Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement profanity.dev as the swear filter with filter notifications. #112

Open
wants to merge 27 commits into
base: staging
Choose a base branch
from
Open

Implement profanity.dev as the swear filter with filter notifications. #112

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
0fff6aa
Start the bridge draft.
benrobson Oct 9, 2024
ba04794
API routes and dashboard UI
benrobson Oct 10, 2024
c58b713
bridge api and dashboard implementation
benrobson Oct 11, 2024
d0d7142
Have target server only execute non-processed tasks
benrobson Nov 9, 2024
dc7f5f3
Add a clean up task and don't execute old tasks.
benrobson Nov 11, 2024
83f42ef
Add bridge discord command permission-gated, add /user/get by discord…
benrobson Nov 13, 2024
4529408
Merge branch 'staging' of https://github.com/ModularSoftAU/zander-web…
benrobson Nov 15, 2024
7133926
Implement api for server sync
benrobson Nov 16, 2024
ffca9dc
Start draft implementation on status command.
benrobson Nov 16, 2024
32e9839
Implement player tracking channel and reimplement /status command.
benrobson Nov 17, 2024
cc5a7d5
Move to once a minute for troubleshooting.
benrobson Nov 17, 2024
2d93f6c
Push for troubleshoot
benrobson Nov 17, 2024
fdbe27d
Deploy revision with profanity.dev
benrobson Nov 19, 2024
d9b5785
Push the Stats update to every 5 minutes.
benrobson Nov 19, 2024
fde62e1
commit
benrobson Nov 19, 2024
d1d5e4e
Change Players online icon
benrobson Nov 19, 2024
7dfec6a
Add an await to the message deletion.
benrobson Nov 19, 2024
12d699b
Tweak score conditions
benrobson Nov 19, 2024
9bef9d2
Merge branch 'staging' of https://github.com/ModularSoftAU/zander-web…
benrobson Jan 19, 2025
a62bb81
Append .js to all imports to be Node v20 compliant.
benrobson Jan 19, 2025
b0b0009
Implement profanity.dev with Filter flag in staff and admin log
benrobson Jan 19, 2025
5325da3
Merge branch 'staging' of https://github.com/ModularSoftAU/zander-web…
benrobson Jan 19, 2025
2703d45
Add profanityData.score >= 1 for testing.
benrobson Jan 20, 2025
e8dcb12
make it only use profanityData.score
benrobson Jan 26, 2025
b450327
Added support for events via guild id
benrobson Feb 11, 2025
1999713
Add a phrasesWhitelist to bypass filter.
benrobson Feb 11, 2025
5a24de1
Remove user check validation
benrobson Feb 11, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions api/common.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,8 @@ import config from "../config.json" assert { type: "json" };
import fetch from "node-fetch";
import { readdirSync } from "fs";
import crypto from "crypto";
import db from "../controllers/databaseController";
import { getWebAnnouncement } from "../controllers/announcementController";
import db from "../controllers/databaseController.js";
import { getWebAnnouncement } from "../controllers/announcementController.js";

/*
Check if a specific feature is enabled.
Expand Down
2 changes: 1 addition & 1 deletion api/internal_redirect/announcement.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { hasPermission, postAPIRequest } from "../common";
import { hasPermission, postAPIRequest } from "../common.js";

export default function announcementRedirectRoute(app, config, lang) {
const baseEndpoint = "/redirect/announcement";
Expand Down
2 changes: 1 addition & 1 deletion api/internal_redirect/application.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { hasPermission, postAPIRequest } from "../common";
import { hasPermission, postAPIRequest } from "../common.js";

export default function applicationRedirectRoute(app, config, lang) {
const baseEndpoint = "/redirect/application";
Expand Down
2 changes: 1 addition & 1 deletion api/internal_redirect/bridge.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { hasPermission, postAPIRequest } from "../common";
import { hasPermission, postAPIRequest } from "../common.js";

export default function bridgeRedirectRoute(app, config, lang) {
const baseEndpoint = "/redirect/bridge";
Expand Down
14 changes: 7 additions & 7 deletions api/internal_redirect/index.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
import applicationRedirectRoute from "./application";
import serverRedirectRoute from "./server";
import webRedirectRoute from "./web";
import announcementsRedirectRoute from "./announcement";
import reportRedirectRoute from "./report";
import vaultRedirectRoute from "./vault";
import bridgeRedirectRoute from "./bridge";
import applicationRedirectRoute from "./application.js";
import serverRedirectRoute from "./server.js";
import webRedirectRoute from "./web.js";
import announcementsRedirectRoute from "./announcement.js";
import reportRedirectRoute from "./report.js";
import vaultRedirectRoute from "./vault.js";
import bridgeRedirectRoute from "./bridge.js";

export default (app, config, lang) => {
applicationRedirectRoute(app, config, lang);
Expand Down
2 changes: 1 addition & 1 deletion api/internal_redirect/report.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { hasPermission, postAPIRequest } from "../common";
import { hasPermission, postAPIRequest } from "../common.js";

export default function reportRedirectRoute(app, config, lang) {
const baseEndpoint = "/redirect/report";
Expand Down
2 changes: 1 addition & 1 deletion api/internal_redirect/server.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { hasPermission, postAPIRequest } from "../common";
import { hasPermission, postAPIRequest } from "../common.js";

export default function serverRedirectRoute(app, config, lang) {
const baseEndpoint = "/redirect/server";
Expand Down
2 changes: 1 addition & 1 deletion api/internal_redirect/vault.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { hasPermission, postAPIRequest } from "../common";
import { hasPermission, postAPIRequest } from "../common.js";

export default function vaultRedirectRoute(app, config, lang) {
const baseEndpoint = "/redirect/vault";
Expand Down
2 changes: 1 addition & 1 deletion api/internal_redirect/web.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { postAPIRequest } from "../common";
import { postAPIRequest } from "../common.js";

export default function webRedirectRoute(app, config, lang) {
const baseEndpoint = "/redirect/web";
Expand Down
7 changes: 6 additions & 1 deletion api/routes/announcement.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,9 @@
import { isFeatureEnabled, required, optional, generateLog } from "../common";
import {
isFeatureEnabled,
required,
optional,
generateLog,
} from "../common.js";

export default function announcementApiRoute(app, config, db, features, lang) {
const baseEndpoint = "/api/announcement";
Expand Down
7 changes: 6 additions & 1 deletion api/routes/application.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,9 @@
import { isFeatureEnabled, required, optional, generateLog } from "../common";
import {
isFeatureEnabled,
required,
optional,
generateLog,
} from "../common.js";

export default function applicationApiRoute(app, config, db, features, lang) {
const baseEndpoint = "/api/application";
Expand Down
103 changes: 101 additions & 2 deletions api/routes/bridge.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,9 @@
import { isFeatureEnabled, required, optional, generateLog } from "../common";
import {
isFeatureEnabled,
required,
optional,
generateLog,
} from "../common.js";

export default function bridgeApiRoute(app, config, db, features, lang) {
const baseEndpoint = "/api/bridge";
Expand Down Expand Up @@ -91,6 +96,100 @@ export default function bridgeApiRoute(app, config, db, features, lang) {
return res;
});

app.post(baseEndpoint + "/clear", async function (req, res) {
isFeatureEnabled(features.bridge, res, lang);

try {
db.query(
`TRUNCATE bridge;`,
function (error, results, fields) {
if (error) {
return res.send({
success: false,
message: `${error}`,
});
}

return res.send({
success: true,
message: `Bridge has now been cleared.`,
});
}
);
} catch (error) {
res.send({
success: false,
message: `${error}`,
});
}

return res;
});
Comment on lines +99 to +127

Check failure

Code scanning / CodeQL

Missing rate limiting High

This route handler performs
a database access
Loading
, but is not rate-limited.

Copilot Autofix AI about 1 month ago

To fix the problem, we need to introduce rate limiting to the route handlers in the bridgeApiRoute function. The best way to achieve this is by using the express-rate-limit middleware. This middleware allows us to set a maximum number of requests a user can make within a specified time window. We will apply this middleware to the relevant routes to ensure they are protected against excessive requests.

We will need to:

  1. Install the express-rate-limit package.
  2. Import the express-rate-limit package in the api/routes/bridge.js file.
  3. Set up a rate limiter with appropriate configuration.
  4. Apply the rate limiter to the relevant routes.
Suggested changeset 2
api/routes/bridge.js

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/api/routes/bridge.js b/api/routes/bridge.js
--- a/api/routes/bridge.js
+++ b/api/routes/bridge.js
@@ -6,2 +6,3 @@
 } from "../common.js";
+import RateLimit from "express-rate-limit";
 
@@ -10,2 +11,10 @@
 
+  // set up rate limiter: maximum of 100 requests per 15 minutes
+  const limiter = RateLimit({
+    windowMs: 15 * 60 * 1000, // 15 minutes
+    max: 100, // max 100 requests per windowMs
+  });
+
+  app.use(limiter);
+
   app.get(baseEndpoint + "/get", async function (req, res) {
@@ -64,3 +73,3 @@
 
-  app.post(baseEndpoint + "/command/add", async function (req, res) {
+  app.post(baseEndpoint + "/command/add", limiter, async function (req, res) {
     isFeatureEnabled(features.bridge, res, lang);
@@ -98,3 +107,3 @@
 
-  app.post(baseEndpoint + "/clear", async function (req, res) {
+  app.post(baseEndpoint + "/clear", limiter, async function (req, res) {
     isFeatureEnabled(features.bridge, res, lang);
@@ -128,3 +137,3 @@
 
-  app.get(baseEndpoint + "/server/get", async function (req, res) {
+  app.get(baseEndpoint + "/server/get", limiter, async function (req, res) {
     isFeatureEnabled(features.bridge, res, lang);
@@ -158,3 +167,3 @@
 
-  app.post(baseEndpoint + "/server/update", async function (req, res) {
+  app.post(baseEndpoint + "/server/update", limiter, async function (req, res) {
     isFeatureEnabled(features.bridge, res, lang);
EOF
@@ -6,2 +6,3 @@
} from "../common.js";
import RateLimit from "express-rate-limit";

@@ -10,2 +11,10 @@

// set up rate limiter: maximum of 100 requests per 15 minutes
const limiter = RateLimit({
windowMs: 15 * 60 * 1000, // 15 minutes
max: 100, // max 100 requests per windowMs
});

app.use(limiter);

app.get(baseEndpoint + "/get", async function (req, res) {
@@ -64,3 +73,3 @@

app.post(baseEndpoint + "/command/add", async function (req, res) {
app.post(baseEndpoint + "/command/add", limiter, async function (req, res) {
isFeatureEnabled(features.bridge, res, lang);
@@ -98,3 +107,3 @@

app.post(baseEndpoint + "/clear", async function (req, res) {
app.post(baseEndpoint + "/clear", limiter, async function (req, res) {
isFeatureEnabled(features.bridge, res, lang);
@@ -128,3 +137,3 @@

app.get(baseEndpoint + "/server/get", async function (req, res) {
app.get(baseEndpoint + "/server/get", limiter, async function (req, res) {
isFeatureEnabled(features.bridge, res, lang);
@@ -158,3 +167,3 @@

app.post(baseEndpoint + "/server/update", async function (req, res) {
app.post(baseEndpoint + "/server/update", limiter, async function (req, res) {
isFeatureEnabled(features.bridge, res, lang);
package.json
Outside changed files

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/package.json b/package.json
--- a/package.json
+++ b/package.json
@@ -51,3 +51,4 @@
     "querystring": "^0.2.1",
-    "zero-md": "^2.3.1"
+    "zero-md": "^2.3.1",
+    "express-rate-limit": "^7.5.0"
   }
EOF
@@ -51,3 +51,4 @@
"querystring": "^0.2.1",
"zero-md": "^2.3.1"
"zero-md": "^2.3.1",
"express-rate-limit": "^7.5.0"
}
This fix introduces these dependencies
Package Version Security advisories
express-rate-limit (npm) 7.5.0 None
Copilot is powered by AI and may make mistakes. Always verify output.
Positive Feedback
Negative Feedback

Provide additional feedback

Please help us improve GitHub Copilot by sharing more details about this comment.

Please select one or more of the options

app.get(baseEndpoint + "/server/get", async function (req, res) {
isFeatureEnabled(features.bridge, res, lang);

try {
db.query(
`SELECT * FROM serverStatus;`,
function (error, results) {
if (error) {
return res.send({
success: false,
message: `Failed: ${error}`,
});
}

return res.send({
success: true,
data: results,
});
}
);
} catch (error) {
return res.send({
success: false,
message: `Unexpected error: ${error}`,
});
}

return res;
});
Comment on lines +129 to +157

Check failure

Code scanning / CodeQL

Missing rate limiting High

This route handler performs
a database access
Loading
, but is not rate-limited.

Copilot Autofix AI about 1 month ago

To fix the problem, we need to introduce rate limiting to the route handlers that perform database access operations. The best way to achieve this is by using the express-rate-limit middleware. This middleware allows us to set a maximum number of requests that can be made to the server within a specified time window. We will apply this middleware to the relevant routes to ensure that they are protected against denial-of-service attacks.

We will need to:

  1. Install the express-rate-limit package.
  2. Import the express-rate-limit package in the file.
  3. Set up a rate limiter with appropriate configuration.
  4. Apply the rate limiter to the relevant routes.
Suggested changeset 2
api/routes/bridge.js

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/api/routes/bridge.js b/api/routes/bridge.js
--- a/api/routes/bridge.js
+++ b/api/routes/bridge.js
@@ -6,4 +6,9 @@
 } from "../common.js";
+import RateLimit from "express-rate-limit";
 
 export default function bridgeApiRoute(app, config, db, features, lang) {
+  const limiter = RateLimit({
+    windowMs: 15 * 60 * 1000, // 15 minutes
+    max: 100, // max 100 requests per windowMs
+  });
   const baseEndpoint = "/api/bridge";
@@ -64,3 +69,3 @@
 
-  app.post(baseEndpoint + "/command/add", async function (req, res) {
+  app.post(baseEndpoint + "/command/add", limiter, async function (req, res) {
     isFeatureEnabled(features.bridge, res, lang);
@@ -98,3 +103,3 @@
 
-  app.post(baseEndpoint + "/clear", async function (req, res) {
+  app.post(baseEndpoint + "/clear", limiter, async function (req, res) {
     isFeatureEnabled(features.bridge, res, lang);
@@ -128,3 +133,3 @@
 
-  app.get(baseEndpoint + "/server/get", async function (req, res) {
+  app.get(baseEndpoint + "/server/get", limiter, async function (req, res) {
     isFeatureEnabled(features.bridge, res, lang);
@@ -158,3 +163,3 @@
 
-  app.post(baseEndpoint + "/server/update", async function (req, res) {
+  app.post(baseEndpoint + "/server/update", limiter, async function (req, res) {
     isFeatureEnabled(features.bridge, res, lang);
@@ -192,3 +197,3 @@
 
-  app.post(baseEndpoint + "/command/process", async function (req, res) {
+  app.post(baseEndpoint + "/command/process", limiter, async function (req, res) {
     isFeatureEnabled(features.bridge, res, lang);
EOF
@@ -6,4 +6,9 @@
} from "../common.js";
import RateLimit from "express-rate-limit";

export default function bridgeApiRoute(app, config, db, features, lang) {
const limiter = RateLimit({
windowMs: 15 * 60 * 1000, // 15 minutes
max: 100, // max 100 requests per windowMs
});
const baseEndpoint = "/api/bridge";
@@ -64,3 +69,3 @@

app.post(baseEndpoint + "/command/add", async function (req, res) {
app.post(baseEndpoint + "/command/add", limiter, async function (req, res) {
isFeatureEnabled(features.bridge, res, lang);
@@ -98,3 +103,3 @@

app.post(baseEndpoint + "/clear", async function (req, res) {
app.post(baseEndpoint + "/clear", limiter, async function (req, res) {
isFeatureEnabled(features.bridge, res, lang);
@@ -128,3 +133,3 @@

app.get(baseEndpoint + "/server/get", async function (req, res) {
app.get(baseEndpoint + "/server/get", limiter, async function (req, res) {
isFeatureEnabled(features.bridge, res, lang);
@@ -158,3 +163,3 @@

app.post(baseEndpoint + "/server/update", async function (req, res) {
app.post(baseEndpoint + "/server/update", limiter, async function (req, res) {
isFeatureEnabled(features.bridge, res, lang);
@@ -192,3 +197,3 @@

app.post(baseEndpoint + "/command/process", async function (req, res) {
app.post(baseEndpoint + "/command/process", limiter, async function (req, res) {
isFeatureEnabled(features.bridge, res, lang);
package.json
Outside changed files

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/package.json b/package.json
--- a/package.json
+++ b/package.json
@@ -51,3 +51,4 @@
     "querystring": "^0.2.1",
-    "zero-md": "^2.3.1"
+    "zero-md": "^2.3.1",
+    "express-rate-limit": "^7.5.0"
   }
EOF
@@ -51,3 +51,4 @@
"querystring": "^0.2.1",
"zero-md": "^2.3.1"
"zero-md": "^2.3.1",
"express-rate-limit": "^7.5.0"
}
This fix introduces these dependencies
Package Version Security advisories
express-rate-limit (npm) 7.5.0 None
Copilot is powered by AI and may make mistakes. Always verify output.
Positive Feedback
Negative Feedback

Provide additional feedback

Please help us improve GitHub Copilot by sharing more details about this comment.

Please select one or more of the options

app.post(baseEndpoint + "/server/update", async function (req, res) {
isFeatureEnabled(features.bridge, res, lang);

const serverInfo = required(req.body, "serverInfo", res);
let lastUpdated = required(req.body, "lastUpdated", res);

try {
const serverInfoString = JSON.stringify(serverInfo);

db.query(
`UPDATE serverStatus SET statusInfo = ?, lastUpdated = ? WHERE serverStatusId = 1;`,
[serverInfoString, lastUpdated],
function (error, updateResults) {
if (error) {
return res.send({
success: false,
message: `Update failed: ${error}`,
});
}

return res.send({
success: true,
message: `Server status updated successfully.`,
});
}
);
} catch (error) {
return res.send({
success: false,
message: `Unexpected error: ${error}`,
});
}
});
Comment on lines +159 to +191

Check failure

Code scanning / CodeQL

Missing rate limiting High

This route handler performs
a database access
Loading
, but is not rate-limited.

Copilot Autofix AI about 1 month ago

To fix the problem, we need to introduce a rate-limiting middleware to the Express application. The express-rate-limit package is a well-known library for this purpose. We will configure the rate limiter to allow a maximum of 100 requests per 15 minutes and apply it to all routes in the bridgeApiRoute function.

  1. Install the express-rate-limit package.
  2. Import the express-rate-limit package in the api/routes/bridge.js file.
  3. Configure the rate limiter with appropriate settings.
  4. Apply the rate limiter to the Express application within the bridgeApiRoute function.
Suggested changeset 2
api/routes/bridge.js

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/api/routes/bridge.js b/api/routes/bridge.js
--- a/api/routes/bridge.js
+++ b/api/routes/bridge.js
@@ -6,4 +6,11 @@
 } from "../common.js";
+import RateLimit from "express-rate-limit";
 
 export default function bridgeApiRoute(app, config, db, features, lang) {
+  const limiter = RateLimit({
+    windowMs: 15 * 60 * 1000, // 15 minutes
+    max: 100, // max 100 requests per windowMs
+  });
+
+  app.use(limiter);
   const baseEndpoint = "/api/bridge";
EOF
@@ -6,4 +6,11 @@
} from "../common.js";
import RateLimit from "express-rate-limit";

export default function bridgeApiRoute(app, config, db, features, lang) {
const limiter = RateLimit({
windowMs: 15 * 60 * 1000, // 15 minutes
max: 100, // max 100 requests per windowMs
});

app.use(limiter);
const baseEndpoint = "/api/bridge";
package.json
Outside changed files

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/package.json b/package.json
--- a/package.json
+++ b/package.json
@@ -51,3 +51,4 @@
     "querystring": "^0.2.1",
-    "zero-md": "^2.3.1"
+    "zero-md": "^2.3.1",
+    "express-rate-limit": "^7.5.0"
   }
EOF
@@ -51,3 +51,4 @@
"querystring": "^0.2.1",
"zero-md": "^2.3.1"
"zero-md": "^2.3.1",
"express-rate-limit": "^7.5.0"
}
This fix introduces these dependencies
Package Version Security advisories
express-rate-limit (npm) 7.5.0 None
Copilot is powered by AI and may make mistakes. Always verify output.
Positive Feedback
Negative Feedback

Provide additional feedback

Please help us improve GitHub Copilot by sharing more details about this comment.

Please select one or more of the options

app.post(baseEndpoint + "/command/process", async function (req, res) {
isFeatureEnabled(features.bridge, res, lang);

Expand All @@ -104,7 +203,7 @@ export default function bridgeApiRoute(app, config, db, features, lang) {
const bridgeApiData = await response.json();

db.query(
`UPDATE bridge SET processed=? WHERE bridgeId=?;`,
`DELETE FROM bridge WHERE bridgeId=?;`,
[1, bridgeId],
function (error, results, fields) {
if (error) {
Expand Down
7 changes: 5 additions & 2 deletions api/routes/discord.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
import { updateAudit_lastMinecraftLogin, updateAudit_lastMinecraftMessage } from "../../controllers/auditController";
import {
updateAudit_lastMinecraftLogin,
updateAudit_lastMinecraftMessage,
} from "../../controllers/auditController.js";
import { Webhook } from "discord-webhook-node";
import { isFeatureEnabled, required } from "../common";
import { isFeatureEnabled, required } from "../common.js";

export default function discordApiRoute(
app,
Expand Down
Loading