Check for 404 before redirecting on no-extension paths

CHANGELOG.md

@@ -47,6 +47,7 @@
     ```sql
     EXECUTE dbo.proc1 DEFAULT
     ```
+- The file-based routing system was improved. Now, requests to `/xxx` redirect to `/xxx/` only if `/xxx/index.sql` exists.
 
 ## v0.35.2
  - Fix a bug with zero values being displayed with a non-zero height in stacked bar charts.

examples/official-site/sqlpage/migrations/64_blog_routing.sql

@@ -0,0 +1,63 @@
+
+INSERT INTO blog_posts (title, description, icon, created_at, content)
+VALUES
+    (
+        'File-based routing in SQLPage',
+        'Understanding how SQLPage maps URLs to files and handles errors',
+        'route',
+        '2025-07-28',
+        '
+SQLPage uses a simple file-based routing system that maps URLs directly to SQL files in your project directory.
+No complex configuration is needed. Just create files and they become accessible endpoints.
+
+This guide explains how SQLPage resolves URLs, handles different file types, and manages 404 errors so you can structure your application effectively.
+
+## How SQLPage Routes Requests
+
+### 1. Site Prefix Handling
+
+If you''ve configured a [`site_prefix`](/your-first-sql-website/nginx) in your settings,
+SQLPage will redirect all requests that do not start with the prefix to `/<site_prefix>`.
+
+### 2. Path Resolution Priority
+
+**Directory requests (paths ending with `/`)**: SQLPage looks for an `index.sql` file in that directory and executes it if found.
+
+**Direct SQL file requests (`.sql` extension)**: SQLPage executes the requested SQL file if it exists.
+
+**Static asset requests (other extensions)**: SQLPage serves files like CSS, JavaScript, images, or any other static content directly.
+
+**Clean URL requests (no extension)**: SQLPage first tries to find a matching `.sql` file. If that doesn''t exist but there''s an `index.sql` file in a directory with the same name, it redirects to the directory path with a trailing slash.
+
+### Error Handling
+
+When, after applying each of the rules above in order, SQLPage can''t find a requested file,
+it walks up your directory structure looking for [custom `404.sql` files](/your-first-sql-website/custom_urls).
+
+## Dynamic Routing with SQLPage
+
+SQLPage''s file-based routing becomes powerful when combined with strategic use of 404.sql files to handle dynamic URLs. Here''s how to build APIs and pages with dynamic parameters:
+
+### Product Catalog with Dynamic IDs
+
+**Goal**: Handle URLs like `/products/123`, `/products/abc`, `/products/new-laptop`
+
+**Setup**:
+```text
+products/
+├── index.sql          # Lists all products (/products/)
+├── 404.sql           # Handles /products/<product-id>
+└── categories.sql    # Product categories (/products/categories)
+```
+
+**How it works**:
+- `/products/` → Executes `products/index.sql` (product listing)
+- `/products/123` → No `123.sql` file exists, so executes `products/404.sql`
+- `/products/laptop` → No `laptop.sql` file exists, so executes `products/404.sql`
+
+**In `products/404.sql`**:
+```sql
+set product_id = substr(sqlpage.path(), 1+length(''/products/''));
+```
+        '
+    );

src/webserver/routing.rs

@@ -1,3 +1,73 @@
+//! This module determines how incoming HTTP requests are mapped to
+//! SQL files for execution, static assets for serving, or error pages.
+//!
+//! ## Routing Rules
+//!
+//! `SQLPage` follows a file-based routing system with the following precedence:
+//!
+//! ### 1. Site Prefix Handling
+//! - If a `site_prefix` is configured and the request path doesn't start with it, redirect to the prefixed path
+//! - All subsequent routing operates on the path after stripping the prefix
+//!
+//! ### 2. Path Resolution (in order of precedence)
+//!
+//! #### Paths ending with `/` (directories):
+//! - Look for `index.sql` in that directory
+//! - If found: **Execute** the SQL file
+//! - If not found: Look for custom 404 handlers (see Error Handling below)
+//!
+//! #### Paths with `.sql` extension:
+//! - If the file exists: **Execute** the SQL file  
+//! - If not found: Look for custom 404 handlers (see Error Handling below)
+//!
+//! #### Paths with other extensions (assets):
+//! - If the file exists: **Serve** the static file
+//! - If not found: Look for custom 404 handlers (see Error Handling below)
+//!
+//! #### Paths without extension:
+//! - First, try to find `{path}.sql` and **Execute** if found
+//! - If no SQL file found but `{path}/index.sql` exists: **Redirect** to `{path}/`
+//! - Otherwise: Look for custom 404 handlers (see Error Handling below)
+//!
+//! ### 3. Error Handling (404 cases)
+//!
+//! When a requested file is not found, `SQLPage` looks for custom 404 handlers:
+//!
+//! - Starting from the requested path's directory, walk up the directory tree
+//! - Look for `404.sql` in each parent directory  
+//! - If found: **Execute** the custom 404 SQL file
+//! - If no custom 404 found anywhere: Return default **404 Not Found** response
+//!
+//! ## Examples
+//!
+//! ```text
+//! Request: GET /
+//! Result: Execute index.sql
+//!
+//! Request: GET /users
+//! - If users.sql exists: Execute users.sql  
+//! - Else if users/index.sql exists: Redirect to /users/
+//! - Else if 404.sql exists: Execute 404.sql
+//! - Else: Default 404
+//!
+//! Request: GET /users/
+//! - If users/index.sql exists: Execute users/index.sql
+//! - Else if users/404.sql exists: Execute users/404.sql  
+//! - Else if 404.sql exists: Execute 404.sql
+//! - Else: Default 404
+//!
+//! Request: GET /api/users.sql  
+//! - If api/users.sql exists: Execute api/users.sql
+//! - Else if api/404.sql exists: Execute api/404.sql
+//! - Else if 404.sql exists: Execute 404.sql  
+//! - Else: Default 404
+//!
+//! Request: GET /favicon.ico
+//! - If favicon.ico exists: Serve favicon.ico
+//! - Else if 404.sql exists: Execute 404.sql
+//! - Else: Default 404
+//! ```
+
 use crate::filesystem::FileSystem;
 use crate::webserver::database::ParsedSqlFile;
 use crate::{file_cache::FileCache, AppState};
@@ -120,9 +190,15 @@ where
         find_file_or_not_found(&path, SQL_EXTENSION, store).await
     } else {
         let path_with_ext = path.with_extension(SQL_EXTENSION);
-        match find_file(&path_with_ext, SQL_EXTENSION, store).await? {
-            Some(action) => Ok(action),
-            None => Ok(Redirect(append_to_path(path_and_query, FORWARD_SLASH))),
+        match find_file_or_not_found(&path_with_ext, SQL_EXTENSION, store).await? {
+            Execute(x) => Ok(Execute(x)),
+            other_action => {
+                if store.contains(&path.join(INDEX)).await? {
+                    Ok(Redirect(append_to_path(path_and_query, FORWARD_SLASH)))
+                } else {
+                    Ok(other_action)
+                }
+            }
         }
     }
 }
@@ -190,7 +266,7 @@ mod tests {
     use std::default::Default as StdDefault;
     use std::path::{Path, PathBuf};
     use std::str::FromStr;
-    use StoreConfig::{Default, Empty, File};
+    use StoreConfig::{Custom, Default, Empty, File};
 
     mod execute {
         use super::StoreConfig::{Default, File};
@@ -332,6 +408,22 @@ mod tests {
 
             assert_eq!(expected, actual);
         }
+
+        #[tokio::test]
+        async fn no_extension_path_that_would_result_in_404_does_not_redirect() {
+            let actual = do_route("/nonexistent", Default, None).await;
+            let expected = custom_not_found("404.sql");
+
+            assert_eq!(expected, actual);
+        }
+
+        #[tokio::test]
+        async fn no_extension_path_that_would_result_in_404_does_not_redirect_with_site_prefix() {
+            let actual = do_route("/prefix/nonexistent", Default, Some("/prefix/")).await;
+            let expected = custom_not_found("404.sql");
+
+            assert_eq!(expected, actual);
+        }
     }
 
     mod not_found {
@@ -402,8 +494,8 @@ mod tests {
     }
 
     mod redirect {
-        use super::StoreConfig::Default;
-        use super::{do_route, redirect};
+        use super::StoreConfig::{Default, Empty};
+        use super::{custom_not_found, default_not_found, do_route, redirect};
 
         #[tokio::test]
         async fn path_without_site_prefix_redirects_to_site_prefix() {
@@ -414,36 +506,42 @@ mod tests {
         }
 
         #[tokio::test]
-        async fn no_extension_and_no_corresponding_file_redirects_with_trailing_slash() {
+        async fn no_extension_and_no_corresponding_file_with_custom_404_does_not_redirect() {
             let actual = do_route("/folder", Default, None).await;
-            let expected = redirect("/folder/");
+            let expected = custom_not_found("404.sql");
 
             assert_eq!(expected, actual);
         }
 
         #[tokio::test]
-        async fn no_extension_no_corresponding_file_redirects_with_trailing_slash_and_query() {
+        async fn no_extension_no_corresponding_file_with_custom_404_does_not_redirect_with_query() {
             let actual = do_route("/folder?misc=1&foo=bar", Default, None).await;
-            let expected = redirect("/folder/?misc=1&foo=bar");
+            let expected = custom_not_found("404.sql");
 
             assert_eq!(expected, actual);
         }
 
         #[tokio::test]
-        async fn no_extension_site_prefix_and_no_corresponding_file_redirects_with_trailing_slash()
-        {
+        async fn no_extension_site_prefix_and_no_corresponding_file_with_custom_404_does_not_redirect(
+        ) {
             let actual = do_route("/prefix/folder", Default, Some("/prefix/")).await;
-            let expected = redirect("/prefix/folder/");
+            let expected = custom_not_found("404.sql");
 
             assert_eq!(expected, actual);
         }
+
+        #[tokio::test]
+        async fn no_extension_returns_404_when_no_404sql_available() {
+            assert_eq!(do_route("/folder", Empty, None).await, default_not_found());
+        }
     }
 
     async fn do_route(path: &str, config: StoreConfig, prefix: Option<&str>) -> RoutingAction {
         let store = match config {
             Default => Store::with_default_contents(),
             Empty => Store::empty(),
             File(file) => Store::new(file),
+            Custom(files) => Store::with_files(&files),
         };
         let config = match prefix {
             None => Config::default(),
@@ -478,6 +576,7 @@ mod tests {
         Default,
         Empty,
         File(&'static str),
+        Custom(Vec<&'static str>),
     }
 
     struct Store {
@@ -512,6 +611,12 @@ mod tests {
             dbg!(&normalized_path, &self.contents);
             self.contents.contains(&normalized_path)
         }
+
+        fn with_files(files: &[&str]) -> Self {
+            Self {
+                contents: files.iter().map(|s| (*s).to_string()).collect(),
+            }
+        }
     }
 
     impl FileStore for Store {
@@ -542,4 +647,80 @@ mod tests {
             Self::new("/")
         }
     }
+
+    mod specific_configuration {
+        use crate::webserver::routing::tests::default_not_found;
+
+        use super::StoreConfig::Custom;
+        use super::{custom_not_found, do_route, execute, redirect, RoutingAction};
+
+        async fn route_with_index_and_folder_404(path: &str) -> RoutingAction {
+            do_route(
+                path,
+                Custom(vec![
+                    "index.sql",
+                    "folder/404.sql",
+                    "folder_with_index/index.sql",
+                ]),
+                None,
+            )
+            .await
+        }
+
+        #[tokio::test]
+        async fn root_path_executes_index() {
+            let actual = route_with_index_and_folder_404("/").await;
+            let expected = execute("index.sql");
+            assert_eq!(expected, actual);
+        }
+
+        #[tokio::test]
+        async fn index_sql_path_executes_index() {
+            let actual = route_with_index_and_folder_404("/index.sql").await;
+            let expected = execute("index.sql");
+            assert_eq!(expected, actual);
+        }
+
+        #[tokio::test]
+        async fn folder_without_trailing_slash_redirects() {
+            let actual = route_with_index_and_folder_404("/folder_with_index").await;
+            let expected = redirect("/folder_with_index/");
+            assert_eq!(expected, actual);
+        }
+
+        #[tokio::test]
+        async fn folder_without_trailing_slash_without_index_does_not_redirect() {
+            let actual = route_with_index_and_folder_404("/folder").await;
+            let expected = default_not_found();
+            assert_eq!(expected, actual);
+        }
+
+        #[tokio::test]
+        async fn folder_with_trailing_slash_executes_custom_404() {
+            let actual = route_with_index_and_folder_404("/folder/").await;
+            let expected = custom_not_found("folder/404.sql");
+            assert_eq!(expected, actual);
+        }
+
+        #[tokio::test]
+        async fn folder_xxx_executes_custom_404() {
+            let actual = route_with_index_and_folder_404("/folder/xxx").await;
+            let expected = custom_not_found("folder/404.sql");
+            assert_eq!(expected, actual);
+        }
+
+        #[tokio::test]
+        async fn folder_xxx_with_query_executes_custom_404() {
+            let actual = route_with_index_and_folder_404("/folder/xxx?x=1").await;
+            let expected = custom_not_found("folder/404.sql");
+            assert_eq!(expected, actual);
+        }
+
+        #[tokio::test]
+        async fn folder_nested_path_executes_custom_404() {
+            let actual = route_with_index_and_folder_404("/folder/xxx/yyy").await;
+            let expected = custom_not_found("folder/404.sql");
+            assert_eq!(expected, actual);
+        }
+    }
 }

tests/errors/mod.rs

@@ -75,13 +75,10 @@ async fn test_default_404_with_redirect() {
     let resp = resp_result.unwrap();
     assert_eq!(
         resp.status(),
-        http::StatusCode::MOVED_PERMANENTLY,
-        "/i-do-not-exist should return 301"
+        http::StatusCode::NOT_FOUND,
+        "/i-do-not-exist should return 404"
     );
 
-    let location = resp.headers().get(http::header::LOCATION).unwrap();
-    assert_eq!(location, "/i-do-not-exist/");
-
     let resp_result = req_path("/i-do-not-exist/").await;
     let resp = resp_result.unwrap();
     assert_eq!(