Skip to content

About

NeuroWebLab backend

Resources

Stars

3 stars

Watchers

2 watching

Forks

Repository files navigation

NeuroWebLab

NeuroWebLab backend

Your portal for collaborative open neuroscience on the web.

Come check out our vision here.

Helpers

Small functions the apps share, exported from require('neuroweblab') (since 1.8.0):

  • escapeRegex(str): user text matched literally in a MongoDB $regex.

  • jsonForScript(value), jsonForAttribute(value): JSON to write into an inline <script> (<, >, &, U+2028, U+2029 escaped), or into a single-quoted attribute (' too).

  • toId(id): the ObjectId of a 24-digit hex string in either case, else null.

  • catchErrors(handler, respond): wraps an async Express handler; an error answers 500 {error: 'Server error'} (or respond(res)) unless headers were sent, and is logged with req.log or console.

  • usernameFromCookie(sessionParser, cookie): the username of the session a WebSocket's cookie belongs to, or null; reads username and the configured usernameField.

  • Saving a project (since 1.9.0, project/save.js): canSaveProject(oldProject, user) (edit on at least one list), keepFieldsWithoutFilesEdit(newProject, oldProject, user, {sanitize}) (the project's own fields need files: edit), saveRefusal() (the 403 body), ignoredChangesMessage(ignored) (the "Some changes (on …) were ignored" text), filesListIncomplete(newProject, oldProject, dropped) (a files list sent before every file was loaded), and since 1.19.0 keepListsWithoutEditRights(newProject, oldProject, user, {rowKey, oldRows}) (each list: longer needs add, shorter remove, any other change, a move included, edit).

  • Validating a project (since 1.10.0, project/rules.js): plainText(str) (markup removed by DOMPurify over jsdom, text kept), sanitize(value) (every string of a JSON value, as plainText), isBlank(v), text(v), listOf(project, type), isBlankRow (blank rows by list), sortRows(type, rows, oldRows, {rowKey, isBlankRow}) (stored, new and blank rows; rowKey is the app's), shortnamePattern (a new shortname: letters, numbers, hyphens), droppedMessage(dropped) (the "Empty rows were not saved" text).

  • The validation core (since 1.20.0, project/validate.js): validateProject(data, {shortname, oldProject, owner, findMissingUsers, rules}) ({project, dropped, stored} or {error}: parse, sanitise, stored and blank rows, shortname, the three lists, field types, collaborator rows, duplicate layer names, anyone, users; the app's rules give collaboratorUser, fileError, layerError, fieldsError, projectError, and optionally rowKey, isBlankRow, checkStoredFields, checkStoredKeys), projectRowProblems(project, rules) (every row problem as if all rows were new, the same checks), duplicateLayerNames(list, stored), parseProjectData(data).

  • Admin access (since 1.11.0, admin/): createAdmins({cfgPath, logger}) (an app's admins list from its cfg.json, with getAdmins, setAdmins, isAdmin, requestUsername, requireAdmin (401/403 JSON) and adminPage({title, userPayload, loginPrompt}) bound to it), requestUsername(req), sendResult(res, action, {logger}) (an admin action's answer, 500 logged), adminLog(db, key, value, {logger, fields}) (an entry in log, returned with its _id; a log line only with a logger).

  • Damaged projects (since 1.23.0, admin/damaged.js, project/versions.js): projectVersion(doc) (a stored version's token: 20 hex digits of the sha256 of its JSON; the admin fixes refuse a project whose token changed since listed, 409) and damagedProjects({sourcesOf, annotatedFiles, save, lockKeys, logger, logFields}) → {findDamagedProjects(db), restoreFiles(db, {shortname, version, save, username}), fixCurrentVersions(db, {shortname, version, username}), isIncompleteSave} (saves that removed files still missing, several current versions; the fixes under the lock, logged with the previous document; the rows put back keep their names), BATCH_LENGTH (100), EXAMPLES (3).

  • Embed (since 1.12.0, embed/, routes/): mintWsTicket(db, scope) (an embed's WebSocket ticket, its scope stored as given, 6 h), findWsTicket(db, token) (the ticket, or null), ensureTicketIndex(db, {logger}) (token_1 and the TTL index expiryDate_1; never rejects), WS_TICKET_COLLECTION, WS_TICKET_DURATION, projectEmbedRoute(adapter) (GET /project/:projectName/embed: 404s, the app's refusal, or 302 to the first file's embed) and firstSource(project).

  • Unknown URLs and errors (since 1.13.0, routes/notFound.js): notFound({view}) (the 404 catch-all: JSON, the error page or plain text), errorHandler({view, isDevelopment, onError}) (the error's status as JSON or the error page; a 5xx message hidden outside development; onError(err, req, res) logs, by default console.error for a 5xx), wantsJSON(req), wantsPage(req), displayPath(req).

  • What a visitor may see (since 1.14.0, project/visible.js, accessControl/): visibleProject(project, user) (a copy: the collaborators other than anyone and modifiedBy, the annotations and the files hidden unless the visitor may view them; missing lists empty), isAnyoneRow(row), collaboratorKey(row) (userID, nickname or username), and AccessControlService.isLoggedOut(user), AccessControlService.getEffectiveAccessLevel(project, user, accessType) (the higher of the own row and anyone; annotations at most view when logged out).

  • Label sets (since 1.15.0, routes/labelSets.js): listLabelSets(dir, {first, logger}) (the *.json files of a folder as [{name, source}], by file name with first before the others; an invalid file skipped and logged) and labelSetsRoute(dir, {first, logger}) (GET /api/getLabelsets, as JSON).

  • Project versions (since 1.16.0, project/versions.js): findCurrentProject(db, shortname, {projection}) (the current version, the newest _id of several; null for none or a non-string shortname), saveProjectVersion(db, doc, {by, keepModified, afterSave}) (inserts a new version, then marks the others backup: true; afterSave({db, previous, version}) after both writes), CURRENT, NEWEST_FIRST. The collection is the configured projectsCollection.

  • Paged files and the project listing (since 1.17.0, routes/projectFiles.js, routes/projectList.js): projectFilesRoute({db, canView, noViewStatus, entries}) (GET /project/json/:projectName/files?start&length[&names=true]: start/length required, clamped to 0 and cut to the list; entries(req, project, slice, {names}) maps the raw slice), projectListRoute({db, canView, row, projection}) (GET /project/json?page: every current project read, filtered by canView, then PROJECTS_PER_PAGE (20) per page as row(req, project)); errors go to next.

  • The project lock (since 1.18.0, project/lock.js): withLock(keys, fn) (runs fn once every earlier holder of each key has finished, FIFO per key; several keys deduplicated and sorted, so no deadlock; a throw releases every key; a bad key is a TypeError), isLocked(key), projectKey(shortname) (project:<shortname>), withProjectLock(shortname, fn). One process, one copy of neuroweblab per app; no timeout.

  • The project routes' access (since 1.21.0, routes/projectAccess.js): visitor(req) (the username, or null logged out; sessions and tokens), noViewStatus(req) (401 logged out, 403 logged in), and five factories, errors to next: projectPageRoute({db, canView, render}) (GET /project/:projectName: 404 Project Not Found, the refusal Not authorized to view project), projectSettingsRoute({db, canView, blankProject, render}) (an unknown project is blankProject(req), render(req, res, project, {isNew})), projectJsonRoute({db, canView, visible}) (404 {error: 'No project <name>'}, ?var=a/b on own properties), projectSaveRoute({lockKeys, save}) and projectDeleteRoute({db, lockKeys, canRemove, remove}) (401 before the lock; the delete's 404 and 403 inside it, {success: false, error, message}); projection (default {_id: 0}) on the four that read.

  • A user's projects and public fields (since 1.22.0, routes/userProjects.js, user/): userProjectsRoute({db, canView, rows, answer}) (GET /user/json/:userName/projects?[start][&length]: the user's current projects, owner or a row by userID, username or nickname, filtered by canView, then sliced; start/length optional, a value that is not digits counts as not given; answer(await rows(req, page)); errors to next), publicUser(doc, usernameField) (the username field and PUBLIC_USER_FIELDS, name, avatarURL, url, joined, own properties only: an allow-list) and publicLoggedUser(reqUser, usernameField) (null logged out, else publicUser plus username).

They live in util/ (and auth/sessionUser.js, project/, admin/, embed/, routes/, user/), each with its spec next to it.

Yearly Node upgrade

All NAAT repos (NeuroWebLab, nwl-components, BrainBox, microdraw) run the current Node LTS, declared in each repo's .nvmrc. The workflow .github/workflows/yearly-node-upgrade.yml runs on the 1st of every month. Each run re-enables the workflow through the GitHub API, so the schedule isn't disabled after 60 days without commits; outside November that's all it does ("keepalive only"). The issue is still yearly: in November it finds the newest Node LTS on nodejs.org, checks that the unofficial linux-x64-glibc-217 builds exist for it (the production servers run CentOS 7), and, if that LTS is newer than .nvmrc, opens an issue labelled node-upgrade with the checklist for the four repos and both servers (deadline: 1 May). It doesn't open a second issue while one is still open, and it only logs a line when the repos are already on the newest LTS.

  • Who gets assigned: set the repository variable NODE_UPGRADE_ASSIGNEES to a comma-separated list of GitHub usernames (Settings → Secrets and variables → Actions → Variables). Assignees must be collaborators of this repo. If the variable is empty, the issue is opened unassigned and says so.
  • Run it by hand: Actions → Yearly Node upgrade → Run workflow. A manual run does the Node check in any month ("Run the Node check" is ticked; untick it for a keepalive only). "Dry run" is ticked by default: the log shows the issue it would open, and nothing is created. Untick it to open the issue now.
  • Locally: NODE_CHECK=true DRY_RUN=true node .github/scripts/yearly-node-upgrade.mjs (add NVMRC=<file containing an older major> to see the issue text).

About

NeuroWebLab backend

Resources

Stars

3 stars

Watchers

2 watching

Forks

Releases

Packages

Used by

Contributors

Languages