Architectural Refactor for Project Structure

[kzndotsh]

Architectural Refactor for Project Structure

The Problem

As Tux has grown, our project structure within the tux/ directory has evolved organically. While functional, it has reached a point where related components are spread out, and the separation between core logic, user-facing features, and utilities is not as clear as it could be. This can increase the cognitive load for new contributors and make maintenance and scaling more difficult in the long run.

The Goal

The purpose of this discussion is to collaboratively decide on a new, more intentional architecture for the project. The ideal structure should:

• Improve maintainability and readability.
• Provide a clear separation of concerns (e.g., backend vs. Discord features).
• Be scalable for future growth (such as adding a web dashboard or API).
• Enhance the overall developer experience.

The Proposals

Below are nine various proposals for restructuring the tux/ directory. They range from simple refactors to patterns inspired by community standards, modern web apps, and formal software architecture. These proposals do not imply automatic consideration because of their existence, they are simply a means of inspiration/brainstorm/example to get the discussion flowing.

Please review each one. They are collapsed for readability.

---

Proposal 1: The "core" and "features" Split

A straightforward approach that divides the application into two main packages: core for backend logic and features for user-facing components.

Structure (Proposal 1)

tux/
├── core/                 # Backend, services, and infrastructure
│   ├── app.py            # (from tux/app.py)
│   ├── bot.py            # (from tux/bot.py)
│   ├── database/         # (from tux/database/*)
│   ├── handlers/         # (from tux/handlers/*)
│   ├── wrappers/         # (from tux/wrappers/*)
│   └── utils/            # (from tux/utils/*)
│
├── features/             # User-facing Discord cogs and UI
│   ├── cogs/             # (from tux/cogs/*)
│   ├── ui/               # (from tux/ui/*)
│   └── help.py           # (from tux/help.py)
│
├── cli/                  # (from tux/cli/*)
├── main.py               # Main application entry point
└── ... (other files)

Pros (Proposal 1)

• Simplicity: Very easy to understand the high-level separation.
• Clear Division: Creates a strong boundary between what is "internal" and "external."

Cons (Proposal 1)

• Generic Naming: features is a bit vague and could become a miscellaneous dumping ground.
• Awkward Dependencies: Shared utilities might be difficult to place without creating circular dependencies or duplication.

---

Proposal 2: The "bot" and "services" Split

This option frames the structure around the two main parts of the application: the bot itself (all Discord-related code) and the backend services that support it.

Structure (Proposal 2)

tux/
├── bot/                  # All Discord-facing logic
│   ├── cogs/             # (from tux/cogs/*)
│   ├── ui/               # (from tux/ui/*)
│   ├── core.py           # (from tux/bot.py)
│   └── help.py           # (from tux/help.py)
│
├── services/             # Backend systems and external API wrappers
│   ├── database/         # (from tux/database/*)
│   ├── wrappers/         # (from tux/wrappers/*)
│   ├── sentry/           # (from tux/utils/sentry_manager.py)
│   └── task_manager/     # (from tux/utils/task_manager.py)
│
├── cli/                  # (from tux/cli/*)
├── utils/                # Shared utilities (from tux/utils/constants.py, exceptions.py, etc.)
├── main.py               # Main application entry point
└── ... (other files)

Pros (Proposal 2)

• Intuitive: The separation between bot and services is conceptually clean and easy to grasp.
• Scalable: New services or bot features have a clear home.
• Shared utils: A dedicated, top-level utils directory solves the problem of common, shared code.

Cons (Proposal 2)

• Slightly More Complex: Introduces one more top-level directory than Proposal 1, which slightly increases complexity.

---

Proposal 3: The Hyper-Specific Functional Split

This is the most granular approach, creating specific top-level directories for each major function of the application. It follows a more rigorous domain-driven design philosophy.

Structure (Proposal 3)

tux/
├── core/                 # The absolute essential components (Bot class, app setup)
│   ├── bot.py            # (from tux/bot.py)
│   └── app.py            # (from tux/app.py)
│
├── discord/              # All direct Discord API interactions and features
│   ├── cogs/             # (from tux/cogs/*)
│   ├── ui/               # (from tux/ui/*)
│   └── handlers/         # (from tux/handlers/*)
│
├── systems/              # Backend logic and external service integrations
│   ├── database/         # (from tux/database/*)
│   ├── wrappers/         # (from tux/wrappers/*)
│   └── tasks/            # (from tux/utils/task_manager.py)
│
├── cli/                  # (from tux/cli/*)
├── utils/                # Shared utilities (from tux/utils/constants.py, etc.)
├── main.py               # Main application entry point
└── ... (other files)

Pros (Proposal 3)

• Extremely Organized: Every component has a very specific, well-named home.
• Highly Scalable: This pattern is used in very large codebases and scales well as complexity grows.

Cons (Proposal 3)

• Over-Structuring: Might be too much for the current size of the project. Can make it harder for new contributors to find things.
• Increased Cognitive Load: More directories to remember and navigate.

---

Proposal 4: The "app" Monolith with "ext"

This model keeps most of the application together in a main app directory but moves all external-facing components (like Discord cogs and the CLI) into an ext (extensions) directory. This is similar to the pattern used by frameworks like Flask or Django.

Structure (Proposal 4)

tux/
├── app/                  # The core application monolith
│   ├── bot.py            # (from tux/bot.py)
│   ├── database/         # (from tux/database/*)
│   ├── handlers/         # (from tux/handlers/*)
│   ├── services/         # (from tux/wrappers/*, tux/utils/task_manager.py)
│   └── utils/            # (from tux/utils/*)
│
├── ext/                  # Extensions that plug into the app
│   ├── cli/              # (from tux/cli/*)
│   ├── cogs/             # (from tux/cogs/*)
│   └── ui/               # (from tux/ui/*)
│
├── main.py               # Main application entry point
└── ... (other files)

Pros (Proposal 4)

• Framework-like: Follows a pattern common in popular web frameworks.
• Clear Entry Points: ext contains all the ways a user or developer "enters" the application.

Cons (Proposal 4)

• Monolithic app: The app directory itself can become large and less organized over time.
• Less Intuitive: The distinction between app and ext may not be as immediately clear as bot vs. services.

---

Proposal 5: The "src" Layout with Domain Grouping

This approach uses a standard "src" layout and groups code by its domain or high-level feature, rather than its technical function.

Structure (Proposal 5)

src/
└── tux/
    ├── __main__.py
    ├── bot.py              # Core bot setup (from tux/bot.py)
    ├── cli.py              # Core CLI setup (from tux/cli/core.py)
    │
    ├── common/             # Shared utilities, database, wrappers
    │   ├── database/       # (from tux/database/*)
    │   └── utils/          # (from tux/utils/*)
    │
    ├── moderation/         # All moderation-related code (cogs, logic, etc.)
    │   ├── cog.py          # (combines files from tux/cogs/moderation/*)
    │   └── services.py     # (logic that might be in the cog today)
    │
    ├── fun/                # Fun commands and features
    │   ├── cog.py          # (combines files from tux/cogs/fun/*)
    │   └── services.py
    │
    └── guild/              # Guild-specific features
        ├── cog.py          # (combines files from tux/cogs/guild/*)
        └── config_view.py  # (from tux/ui/views/config.py)

Pros (Proposal 5)

• Feature-Oriented: Organizes code around what it does for the user, making it easy to work on a single feature.
• Encapsulation: Keeps all code related to a feature (Discord commands, backend logic, UI) in one place.

Cons (Proposal 5)

• Unconventional: This is a less common pattern and might be confusing for developers used to technical layering (/cogs, /services).
• Potential for Duplication: Common logic might be duplicated across feature packages if not carefully managed in common.

---

Proposal 6: The "Web App" / Next.js Inspired Structure

This model borrows concepts from modern web frameworks like Next.js, organizing the project by features ("routes") and co-locating related code. It emphasizes a strong separation between feature logic, reusable UI components, and shared libraries.

Structure (Proposal 6)

tux/
├── app/                  # "Routes" or features, like Next.js's app router
│   ├── moderation/       # Each feature is a self-contained unit
│   │   ├── commands.py   # (from tux/cogs/moderation/*)
│   │   ├── views.py      # (specific views, e.g. from tux/ui/views/tldr.py if it were a command)
│   │   └── actions.py    # (logic currently inside the command functions)
│   │
│   └── ... (other cogs as feature folders)
│
├── components/           # Reusable UI components (like React components)
│   ├── embeds.py         # (from tux/ui/embeds.py)
│   ├── views.py          # (generic views, e.g. tux/ui/views/confirmation.py)
│   └── modals.py         # (from tux/ui/modals/*)
│
├── lib/                  # Shared libraries and utilities
│   ├── database/         # (from tux/database/*)
│   ├── wrappers/         # (from tux/wrappers/*)
│   └── utils/            # (from tux/utils/*)
│
├── core/                 # Core application setup (the "root layout")
│   ├── bot.py            # (from tux/bot.py)
│   ├── app.py            # (from tux/app.py)
│   └── handlers/         # (from tux/handlers/*)
│
├── cli/                  # (from tux/cli/*)
├── main.py
└── ...

Pros (Proposal 6)

• Modern & Scalable: This pattern is popular in web development for a reason; it scales very well and is intuitive for developers familiar with these frameworks.
• Co-location: Keeping feature-specific logic, commands, and UI together makes development on a single feature much faster and easier to reason about.
• Clear Separation: There's a very strong distinction between features (app), reusable UI (components), shared logic (lib), and application setup (core).

Cons (Proposal 6)

• Unconventional for Bots: This is not a typical structure for a Discord bot project. It might require more initial setup and explanation for contributors unfamiliar with web development patterns.
• Potential for Boilerplate: Each new feature in app/ might require creating several files (commands.py, views.py, etc.), which could feel repetitive.

---

Proposal 7: The Hybrid "Monorepo-Ready" Structure (Community Inspired)

This proposal is a synthesis of the most common and scalable patterns observed across dozens of other major Discord bots, combined with modern architectural practices. It is designed to be immediately familiar, highly organized, and ready to scale into a full-stack application (e.g., with a website or API).

Structure (Proposal 7)

tux/
├── bot/                  # Contains all code for the Discord bot application.
│   │                     # This makes it easy to add `website/` or `api/` later.
│   ├── core/             # Essential bot setup, client, global checks, and handlers.
│   │   ├── client.py     # (from tux/bot.py)
│   │   ├── checks.py     # (from tux/utils/checks.py)
│   │   └── handlers.py   # (from tux/handlers/event.py, error.py)
│   │
│   ├── features/         # Replaces 'cogs'. Holds all user-facing commands, grouped by feature.
│   │   ├── moderation/   # (from tux/cogs/moderation/ban.py, kick.py, etc.)
│   │   ├── fun/          # (from tux/cogs/fun/fact.py, etc.)
│   │   └── utility/      # (from tux/cogs/utility/ping.py, afk.py, etc.)
│   │
│   ├── components/       # Reusable UI components (embeds, views, modals).
│   │   ├── embeds.py     # (from tux/ui/embeds.py)
│   │   └── views.py      # (from tux/ui/views/*)
│
├── services/             # Backend logic, external API wrappers, database clients.
│   │                     # These are kept separate as they could be used by other apps (website, etc).
│   ├── database/         # (from tux/database/*)
│   ├── sentry/           # (from tux/utils/sentry_manager.py)
│   └── wrappers/         # (from tux/wrappers/*)
│
├── shared/               # Python code shared by ALL applications (bot, cli, future web).
│   ├── constants.py      # (from tux/utils/constants.py)
│   ├── exceptions.py     # (from tux/utils/exceptions.py)
│   └── types.py          # (new file for shared data types)
│
├── cli/                  # (from tux/cli/*)
└── main.py

Pros (Proposal 7)

• Community Standard: Aligns closely with the successful patterns seen in other large bots, making it easier for new contributors to understand.
• Monorepo-Ready: The top-level bot/ directory provides a clean namespace, making it trivial to add a website/ or api/ directory at the same level in the future.
• Extremely Clear Separation: The roles of bot/, services/, and shared/ are unambiguous.
• Clean Feature Development: Working on a feature in bot/features/ is self-contained, but it has easy access to reusable UI from bot/components/ and logic from services/.

Cons (Proposal 7)

• Slightly More Verbose Imports: Imports might be slightly longer (e.g., from bot.features import moderation), but this explicitness is generally considered a good thing for clarity.

---

Proposal 8: The "Clean/Hexagonal" Architecture

This is a highly disciplined, academic approach based on patterns like Clean Architecture. The goal is maximum separation of concerns, making the core application logic completely independent of external frameworks (like Discord.py) and services (like the database). The "core" of the application knows nothing about the "outside world."

Structure (Proposal 8)

tux/
├── domain/                 # The absolute core. Pure Python objects and business rules.
│   │                       # NO discord.py, database, or API logic here.
│   ├── moderation/
│   │   ├── entities.py       # e.g., A `Case` dataclass (replaces prisma model)
│   │   └── repository.py     # Defines an *interface* for how to save a `Case`
│   │
│   └── ...
│
├── application/            # The "use cases." Orchestrates the domain logic.
│   ├── services/
│   │   └── moderation.py   # `ModerationService` with `ban_member()` logic
│   │
│   └── dto.py              # e.g., `BanMemberDTO` for carrying data from discord to service
│
├── infrastructure/         # The "outside world." Frameworks, drivers, and tools.
│   ├── discord/            # Adapter for Discord
│   │   ├── cogs/           # e.g., `moderation.py` (from tux/cogs/moderation/ban.py)
│   │   ├── bot.py          # The bot instance (from tux/bot.py)
│   │   └── ui/             # (from tux/ui/*)
│   │
│   ├── database/           # Adapter for the database
│   │   └── repositories/   # *Implements* the domain repository with prisma-client-py
│   │
│   └── cli/                # Adapter for the CLI (from tux/cli/*)
│
└── main.py                   # Wires everything together (Dependency Injection).

How it Works (Proposal 8)

1. A command in infrastructure/discord/cogs/moderation.py is invoked.
2. It calls a method in the application/services/moderation.py service.
3. The service uses pure domain/moderation/entities.py objects and calls methods on a repository interface defined in domain/moderation/repository.py.
4. The actual database work is done by the concrete implementation in infrastructure/database/repositories/, which was "injected" into the service when the bot started.

Pros (Proposal 8)

• Maximum Decoupling: The core logic can be reused if you ever switch frameworks or add a gRPC API.
• Supreme Testability: Unit test the application layer without mocking Discord or database objects.
• Enforced Boundaries: Impossible to accidentally mix database code with command logic.

Cons (Proposal 8)

• Very High Complexity & Boilerplate: This is a heavyweight pattern for large, enterprise-scale applications.
• Unconventional for Bots: Steep learning curve for new contributors.
• Potential Over-Engineering: Benefits may not be realized without concrete plans to reuse the core logic on multiple platforms.

---

Proposal 9: The "Self-Contained Packages" (Modular Plugins) Structure

This approach treats every feature of the bot as a complete, self-contained Python package. The main application is merely a lightweight shell responsible for discovering and loading these "plugins."

Structure (Proposal 9)

tux/
├── packages/             # A directory containing all the installable feature packages
│   ├── moderation/
│   │   ├── __init__.py   # Defines the package's public API and a `setup` function
│   │   ├── commands.py   # (from tux/cogs/moderation/*)
│   │   ├── services.py   # (new file for logic currently in cogs)
│   │   └── models.py     # (conceptual mapping from prisma/schema/moderation.prisma)
│   │
│   └── ...
│
├── core/                 # The application shell and shared infrastructure
│   ├── bot.py            # The bot instance; discovers and runs `setup` from each package
│   ├── database.py       # (from tux/database/client.py)
│   ├── ui/               # (generic UI like tux/ui/views/confirmation.py)
│   └── utils/            # (generic utils from tux/utils/*)
│
├── cli.py                # (from tux/cli/core.py)
└── main.py

How it Works (Proposal 9)

1. On startup, the core/bot.py scans the packages/ directory.
2. For each package (e.g., moderation), it imports it and calls a well-defined entry point function, like setup(bot).
3. The moderation package's setup function is responsible for adding its own cogs, views, and listeners to the bot instance. It gets any dependencies like a database session from the bot.

Pros (Proposal 9)

• Ultimate Modularity & Reusability: Features are truly independent and could be shared between bots.
• Clear Ownership & Boundaries: Perfect for larger teams to own entire features.
• Extensibility: Adding a new feature is as simple as dropping a new, compliant package into the packages directory.

Cons (Proposal 9)

• High Initial Overhead: Requires a more complex package loading and dependency injection mechanism.
• Discovery Challenges: Finding where functionality lives means first identifying the responsible package.
• Cross-Package Communication: Direct communication between packages can be complex.

---

Request for Feedback

We invite all maintainers and contributors to review these proposals and provide your thoughts. To help guide the discussion, please consider the following:

1. Which proposal(s) do you prefer, and why?
2. Which proposal(s) do you dislike, and why?
3. Are there any modifications you would suggest to one of the existing proposals?
4. What are the most important factors for you in a project structure (e.g., ease of finding files, scalability, low complexity, contributor friendliness)?

Your feedback is crucial for setting a clear direction for the future of Tux. Thank you!

[arutonee1]

I think in general, more structure is better.* I personally find it easier to find things in a project when I don't have to guess as to where it is, and one of the easiest ways to achieve this is with a highly structured hierarchical setup.

If we are optimizing for approachability from a new contributor perspective, I think it would make the most sense to decouple technical details from the structuring as much as possible, but the way described in the "src" method is way too prone to duplication and general confusion, especially since there's already a wealth of unused or duplicated code in the current codebase. This method would also make it a bit harder to make "drop-in" features- I personally think one should be able to derive a single class or apply a single decorator and have the feature just work, without any extra poking around.

A combination of the self-contained packages (9) and hyper-specific split (3) is what I would lean towards, though there are definitely examples of the other structures working out if designed properly. The core/features split (1) also looks appealing, but may need some care when implementing.

* Of course, to a certain degree- it's very easy to get caught up in restructuring and forget to write any meaningful code. I do still think that being as specific as possible is definitely worth it though ^w^