LupiNexMedia/LupiNex-Connect
1
0
Fork 0
Code Actions Activity

Initial commit

Browse Source
This commit is contained in:
Andreas Reimann
2026-08-30 15:23:57 +02:00
commit dab5679098
1293 changed files with 240952 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.backup/docs/backend-helper.md
+576
View File
@@ -0,0 +1,576 @@
- [Service Manager](#service-manager)
- [Available methods](#available-methods)
- [Available services](#available-services)
- [Usage](#usage)
- [MySQL Service](#mysql-service)
- ["Service Manager" usage](#service-manager-usage)
- [Database \& Role Mapping (Overview)](#database--role-mapping-overview)
- [Using the pools](#using-the-pools)
- [WebSocket Service](#websocket-service)
- ["Service Manager" usage](#service-manager-usage-1)
- [Connection Manager](#connection-manager)
- [Using the WebSocket Connection Manager](#using-the-websocket-connection-manager)
- [Heartbeat system](#heartbeat-system)
- [WebSocket communication format](#websocket-communication-format)
- [Scheduler Service](#scheduler-service)
- ["Service Manager" usage](#service-manager-usage-2)
- [Scheduler Memory](#scheduler-memory)
- [Task structure](#task-structure)
- [Usage](#usage-1)
- [Logging Service](#logging-service)
- ["Service Manager" usage](#service-manager-usage-3)
- [Logger Service Internals](#logger-service-internals)
- [Writing logs](#writing-logs)
- [Read Logs](#read-logs)
- [Authentication](#authentication)
- [Available functions](#available-functions)
- [Usage](#usage-2)
---
## Service Manager
### Available methods
- start
- stop
- restart
- status
### Available services
- mysql
- websocket
- logger
- scheduler
### Usage
**Note:** All service functions are async, use `await` when calling them.
See the documentation of the services themselves below for further information on how to use the functions.
```js
const services = require('./services/manager.js'); // from the ROOT-Directory of the service
// start a service
const result = await services.start.mysql();
console.log(result);
// stop a service
const result = await services.stop.websocket();
console.log(result);
// restart a service
const result = await services.restart.logger();
console.log(result);
// get status of a service
const status = await services.status.scheduler();
console.log(status);
```
---
## MySQL Service
The MySQL service is started via the service manager and provides connection pools for each database, method (read/write/delete/backup) and role (everyone/auth/system).
### "Service Manager" usage
```js
const services = require('./services/manager.js'); // from the ROOT-Directory of the service
// Start the mysql service and get the pools
const mysqlData = await services.start.mysql();
console.log(mysqlData.online); // true if started successfully
const pools = require('./services/database/pools.js').get(); // from the ROOT-Directory of the service
// Stop the mysql service
const stopResult = await services.stop.mysql(pools);
console.log(stopResult.online); // false if all pools were closed
// Restart the mysql service
const restartResult = await services.restart.mysql(pools);
console.log(restartResult.online);
// Check status
const status = await services.status.mysql(pools);
console.log(status);
/* Example output:
{
dbName1: { r: { e: { online: true }, a: { online: true }, s: { online: true } }, ... },
dbName2: { ... }
}
*/
```
### Database & Role Mapping (Overview)
- **Databases**
- name
- name1
- **Methods**
- r = read
- w = write
- d = delete
- b = backup
- **Roles**
- e = everyone
- a = auth
- s = system
> For the full configuration, see `ROOT/service.backend/mapping.js`
### Using the pools
```js
const pools = require('./services/database/pools.js').get(); // from the ROOT-Directory of the service
// Example: read (r) from a table in db "name" with auth (a) role
const pool = pools.name.r.a;
const [rows] = await pool.query('SELECT * FROM users WHERE active = 1');
console.log(rows);
// Example: write (w) to a table in db "name1" with system (s) role
const writePool = pools.name1.w.s;
await writePool.query('INSERT INTO logs (event) VALUES (?)', ['Test']);
// Example: delete (d) rows from a table in db "name" with auth (a) role
const deletePool = pools.name.d.a;
await deletePool.query('DELETE FROM logs WHERE created_at < NOW() - INTERVAL 30 DAY');
// Example: backup (b) method with system (s) role
const backupPool = pools.name1.b.s;
const [backupData] = await backupPool.query('SELECT * FROM important_table');
console.log(backupData);
```
---
## WebSocket Service
The WebSocket service is started via the service manager.
### "Service Manager" usage
```js
const services = require('./services/manager.js'); // from the ROOT-Directory of the service
// Start the websocket service
const wsData = await services.start.websocket();
console.log(wsData.online); // true if started successfully
// Stop the websocket service
const stopResult = await services.stop.websocket();
console.log(stopResult.online);// false if stopped successfully
// Restart the websocket service
const restartResult = await services.restart.websocket();
console.log(restartResult.online);
// Check status
const status = await services.status.websocket();
console.log(status);
/* Example output:
{
online: true,
clients: {
"connectionId1": {
metadata: { connectedAt: 1678321234567 },
lastHeartbeat: 1678321240000
}
}
}
*/
```
**Note**: Restarting the WebSocket service (`services.restart.websocket()`) reloads all actions, including any changes or new files in subfolders.
### Connection Manager
Active WebSocket connections are managed centrally via `services/websocket/connections.js`.
The connection manager:
- Stores all active connections
- Assigns a unique ID to each client
- Tracks heartbeat timestamps
- Automatically removes disconnected or stale clients
- Provides broadcast functionality
This works similarly to the MySQL pool manager:
Connections are stored once and can be accessed globally without restarting the service.
#### Using the WebSocket Connection Manager
```js
const connections = require('./services/websocket/connections.js') // from the ROOT-Directory of the service
// Suppose you have a new WebSocket client "ws"
const clientMetadata = { username: 'Username' };
// 1. Add the client to the manager
const connectionId = connections.add(ws, clientMetadata);
console.log('New connection ID:', connectionId);
// 2. Start the heartbeat mechanism to keep connections alive (every 30 seconds)
connections.startHeartbeat();
// 3. Broadcast a message to all clients
connections.broadcast({ action: 'notify', message: 'Server is running!' });
// 4. Get a specific connection's info
const clientData = connections.get(connectionId);
console.log(clientData);
// 5. Get all active connections
const allClients = connections.getAll();
console.log(allClients);
// 6. Remove a client (e.g., when they disconnect)
connections.remove(connectionId);
// 7. Stop the heartbeat when shutting down the server
connections.stopHeartbeat();
// 8. Clear all connections (e.g., when stopping the WebSocket service)
connections.clearAll();
```
### Heartbeat system
The service automatically sends periodic ping frames to all clients.
If a client does not respond within two intervals, the connection is removed automatically.
This ensures that stale or broken connections are cleaned up without manual intervention.
### WebSocket communication format
After a client connects to the WebSocket server, all communication must follow this message format:
```json
{
action: "",
data: { ... }
}
```
- action: string that matches a loaded action file (ROOT/websocket)
- data: payload passed to the action handler
Example:
```json
{
"action": "user.getUserData", // in this case, we have the action file at 'ROOT/websocket/user/getUserData.js'
"data": { "userId": 42 }
}
```
All the actions are stored in the "websocket" directory of this service. The system automatically loads all .js files inside this folder and all subfolders recursively. Action names are generated from the folder structure using dot-notation.
Example:
- websocket/user/getUserData.js → "user.getUserData"
- websocket/system/internal/restart.js → "system.internal.restart"
Each action file must export an async function:
```js
module.exports = async (data, ws, connectionId) => {
// data = payload from client
// ws = WebSocket instance
// connectionId = internal connection ID
}
// Detailed example:
module.exports = async (data, ws, connectionId) => {
try {
console.log(`TestAction triggered by connection ${connectionId}`);
console.log('Received data:', data);
// set response object
const response = {
message: 'TestAction executed successfully!',
receivedData: data,
timestamp: Date.now(),
connectionId
};
// send response back to the client
ws.send(JSON.stringify({
action: 'test.testaction.response', // optional: own response action
data: response
}));
return response;
} catch (err) {
console.error('Error in TestAction:', err);
ws.send(JSON.stringify({
action: 'test.testaction.error', // optional: own response action
data: { message: err.message }
}));
throw err;
}
};
```
On the client side it looks like this:
```js
// Connect to server
const ws = new WebSocket('ws://localhost:3001');
// Send a message
ws.send(JSON.stringify({
action: 'test.testaction',
data: { foo: 'bar', num: 42 }
}));
// Get a message
ws.onmessage = (msg) => {
const parsed = JSON.parse(msg.data);
console.log('Server response:', parsed);
};
```
---
## Scheduler Service
The Scheduler service is started via the service manager. It manages tasks that can be scheduled to run once at a specific time or repeatedly at defined intervals.
### "Service Manager" usage
```js
const services = require('./services/manager.js'); // from the ROOT-Directory of the service
// Start the scheduler service
const schedulerData = await services.start.scheduler();
console.log(schedulerData.online); // true if started successfully
const tasks = schedulerData.tasks;
// Stop the scheduler service
const stopResult = await services.stop.scheduler();
console.log(stopResult.online); // false if all tasks were stopped
// Restart the scheduler service
const restartResult = await services.restart.scheduler();
console.log(restartResult.online);
// Check status
const status = await services.status.scheduler();
console.log(status);
/* Example output:
{
"taskName1": { active: true, nextRun: 1678325400000, lastRun: 1678321800000, error: null },
"taskName2": { active: false, nextRun: null, lastRun: 1678320000000, error: 'Some error' }
}
*/
```
### Scheduler Memory
The scheduler memory holds all task states centrally. All other modules (start, stop, status) use it to manage tasks.
#### Task structure
```json
{
"taskName": {
"active": true, // whether the task is currently running
"oneTime": false, // true = runs only once, false = repeats
"intervalInSec": 3600, // interval in seconds for repeating tasks
"lastRun": null, // timestamp of last execution
"nextRun": null, // timestamp of next execution
"error": null, // last error if task failed
"action": "taskName" // name of the JS file in ROOT/tasks to execute
}
}
```
**Note**:
- The tasks object in memory contains all loaded tasks.
- The timer property exists internally in memory but is not stored in JSON.
- When a task is active, its timer starts automatically. One-time tasks (oneTime: true) execute once after intervalInSec, while recurring tasks (oneTime: false) repeat indefinitely at the specified interval.
- Each task executes the function exported by its action file in ROOT/tasks. Errors are saved in task.error
- The functions in the tasks folder should be `async` to ensure proper execution of asynchronous operations.
#### Usage
```js
const memory = require('./services/scheduler/memory.js'); // from the ROOT-Directory of the service
// Load all tasks from tasks.json
// Recurring tasks (oneTime: false) are started automatically
// One-time tasks with active: true also start automatically
memory.loadTasks();
// Get a single task
const task = memory.getTask('updater');
console.log(task);
// Get all tasks
const allTasks = memory.getAllTasks();
console.log(allTasks);
// Add or update a task
// If active: true, the timer starts immediately; if oneTime: true, task executes only once after interval
memory.setTask('cleanup', {
active: true,
oneTime: true,
intervalInSec: 10, // in this case, the interval will set delay
action: 'cleanup'
});
// Remove a task
memory.removeTask('cleanup');
```
**Note**:
- The memory object is shared globally. Any changes you make here immediately affect the scheduler service.
- Automatic execution, lastRun/nextRun tracking, and error logging are all handled internally.
---
## Logging Service
The Logging service is started via the service manager. It provides a central way to write log data into the logging database, using the system write pool. The service can be started, stopped, restarted, and its status can be checked. When offline, no log entries are written.
### "Service Manager" usage
```js
const services = require('./services/manager.js'); // from the ROOT-Directory of the service
// Start the logger service
const loggerData = await services.start.logger();
console.log(loggerData.online); // true if started successfully
const tasks = loggerData.tasks;
// Stop the logger service
const stopResult = await services.stop.logger();
console.log(stopResult.online); // false if stopped
// Restart the logger service
const restartResult = await services.restart.logger();
console.log(restartResult.online);
// Check status
const status = await services.status.logger();
console.log(status);
/* Example output:
{ "online": true }
*/
```
### Logger Service Internals
- The logger service maintains a single `online` flag to determine whether logging is active. This is checked before every write to ensure no logs are written while offline.
- When the service is stopped, the `online` flag is set to `false`, preventing any writes to the database.
- Log entries are written dynamically to tables in the `logging` database via the `write()` function.
- The MySQL write pool (`w.s`, `write` method → `system` role) is used internally to insert data.
#### Writing logs
Logs are written using the exported `write()` function from `ROOT/service.backend/services/logger/writeLogs.js`. This function dynamically maps an object of key/value pairs (`logData`) to columns in a specified table. The service must be online for writes to succeed.
```js
const logger = require('./services/logger/writeLogs.js'); // from the ROOT-Directory of the service
module.exports = async (data, ws, connectionId) => {
try {
// Decide internally which table to log to
const tableName = 'access_logs';
// Write log entry (log data can be specified via data.logData or any other object)
const result = await logger.write(data, tableName);
console.log('Log write result:', result);
// Send response back to WebSocket client as needed
} catch (err) {
console.error('Error in TestAction:', err);
throw err;
}
};
```
**Note**:
- If the logger service is offline (`online: false`), the `write()` function returns `{ success: false, error: 'Logger is inactive' }`.
- Columns in `logData` must match the table schema. Values are dynamically inserted into placeholders to prevent SQL injection. Keys not present in the table will be ignored or cause an error depending on DB configuration.
- Any log type (access, error, event, custom) can be handled with the same `write()` function. The table name determines the type.
#### Read Logs
```js
const loggerRead = require('./services/logger/readLogs.js'); // from the ROOT-Directory of the service
// 1. Get all logs from a table
const allLogs = await loggerRead.getAll('access_logs');
console.log(allLogs);
// 2. Get filtered logs from a table
const filteredLogs = await loggerRead.get('access_logs', { user: 'user', action: 'login' });
console.log(filteredLogs);
```
---
## Authentication
The authentication service handles user registration, login, account updates, and email-based account activation. It provides secure password hashing, session management via JWT, and role assignment.
### Available functions
- register
- login
- update
- activate
- check
### Usage
```js
const auth = require('./middleware/auth.js'); // from the ROOT-Directory of the service
// Register a new user
const newUser = await auth.register({
userEmail: 'user@example.com',
userName: 'myUsername',
userPass: 'strongPassword',
userPicture: '/path/to/picture.png', // optional
fromProject: 'myProject'
});
console.log(newUser);
/* Example output:
{
userId: 'uuid-v4',
authKey: 'generated-auth-key',
sessionKey: 'jwt-session-token',
userName: 'myUsername',
userEmail: 'user@example.com',
userPicture: 'uuid-v4.png',
fromProject: 'myProject',
assignedRoles: ['default']
}
*/
// Login
const loggedIn = await auth.login({
userLogin: 'user@example.com', // or username
userPass: 'strongPassword',
sessionKey: null // optional: existing session key for auto-login
});
console.log(loggedIn);
// Update user information
const updatedUser = await auth.update({
userId: loggedIn.userId,
authKey: loggedIn.authKey,
newUserName: 'newUsername', // optional
newUserEmail: 'newEmail@example.com', // optional
newUserPass: 'newPassword', // optional
newUserPicture: '/path/to/new.png', // optional
assignedRoles: ['default', 'activated'], // optional
additionalPermissions: ['admin'], // optional
state: 'active' // optional
});
console.log(updatedUser);
// Activate account (after receiving activation email)
const activation = await auth.activate({
activationToken: '<JWT-TOKEN-FROM-EMAIL>'
});
console.log(activation);
/* Example output:
{
message: 'Account successfully activated.',
assignedRoles: ['default', 'activated']
}
*/
```
**Note**:
- *Passwords* are hashed with bcrypt using a configurable number of salt rounds (`SALT_ROUNDS` in `.env`).
- *Sessions* are managed with JWTs and expire according to `JWT_EXPIRES`.
- *User pictures* are resized to `250x250px` PNGs automatically and stored in the user picture directory.
- *Roles* are stored as JSON arrays in the database and include a default role `'default'` for all new users.
- *Activation* requires the user to click the link sent via email. Once activated, the role `'activated'` is added automatically.
.backup/docs/index.md
+65
View File
@@ -0,0 +1,65 @@
# Layer -1 ("Layer minus 1") – Data Center ([*← Zurück*](../README.md))
Das **Data Center** (im Folgenden als "**L-1**" bezeichnet) ist das Herzstück des **"LNX-Connect"-Ökosystems** (im Folgenden "**LCE**"; "*E*" für "Ecosystem"). **L-1** ist das einzige Modul, das Zugriff auf die Server-interne Datenbank erhält und nur Anfragen vom **Gateway** (im Folgenden "**L0**") akzeptiert und verarbeitet.
---
<h2>Inhaltsverzeichnis</h2>
- [Layer -1 ("Layer minus 1") – Data Center (*← Zurück*)](#layer--1-layer-minus-1--data-center--zurück)
- [Definition](#definition)
- [Aufgaben und Verantwortlichkeiten](#aufgaben-und-verantwortlichkeiten)
- [Kommunikation und Zugriffskontrolle](#kommunikation-und-zugriffskontrolle)
- [Datenverarbeitung](#datenverarbeitung)
- [Datenintegrität und Validierung](#datenintegrität-und-validierung)
- [Fehlerbehandlung](#fehlerbehandlung)
- [Module](#module)
---
## Definition
Neben der Datenbank ([*MySQL*](https://www.mysql.com/de/)) ist **L-1** das zentrale Herzstück des **LCE**. Durch eine implementierte Sicherheitsebene mittels [JWT](https://www.npmjs.com/package/jsonwebtoken) sowie weiterer sicherheitsrelevanter Mechanismen wird sichergestellt, dass ausschließlich **L0** Zugriff auf diesen Layer erhält.
## Aufgaben und Verantwortlichkeiten
Da **L-1** der einzige Layer mit Zugriff auf die interne Datenbank ist, besteht die Aufgabe dieses Layers darin, die in der Datenbank gespeicherten Datensätze zu verwalten sowie deren Verarbeitung zu steuern. Die folgenden Aufgabenbereiche gehören dazu:
- Authentifizierung der Anfrage bzw. des Anfragenstellers
- Verarbeitungsschritte nach Authentifizierung:
- Überprüfung der empfangenen Datensätze auf Fehler
- Abgleich der gelieferten Daten mit der Datenbank
- Senden bzw. Schreiben von angefragten oder gelieferten Daten
## Kommunikation und Zugriffskontrolle
**L-1** wird ausschließlich von **L0** über eine interne WebSocket-Verbindung angesprochen, welcher auf `127.0.0.1` lauscht. Dadurch wird sichergestellt, dass kein externer Zugriff direkt auf die Datenbank erfolgen kann. Sämtliche Anfragen müssen den gesicherten Weg über **L0** durchlaufen, wo diese vorab authentifiziert und validiert werden.
Die Kommunikation erfolgt nach einem Request-/Response-Prinzip, wobei **L-1** ausschließlich *reaktiv* arbeitet und keine eigenen Anfragen initiiert.
## Datenverarbeitung
Die Datenverarbeitung in **L-1** erfolgt strikt nach einem definierten Ablauf. Jede eingehende Anfrage wird zunächst auf Gültigkeit geprüft und anschließend entsprechend ihres Typs verarbeitet. Dabei wird zwischen Lese-, Schreib- und Löschoperationen (*CRUD*; Create, Read, Update, Delete) unterschieden.
Nach erfolgreicher Authentifizierung und Validierung werden die angeforderten Operationen auf der Datenbank ausgeführt. Anschließend wird eine entsprechende Antwort generiert und an **L0** zurückgesendet, welcher die Antwort dann an den jeweiligen Client weiterleitet. Dieser strukturierte Ablauf stellt sicher, dass all Datenoperationen kontrolliert und nachvollziehbar durchgeführt werden.
## Datenintegrität und Validierung
**L-1** stellt sicher, dass alle verarbeiteten Daten konsistent und fehlerfrei bleiben. Eingehende Datensätze werden vor der Verarbeitung auf Struktur, Vollständigkeit und mögliche Fehler überprüft.
Durch den Abgleich mit bestehenden Daten in der Datenbank wird verhindert, dass inkonsistente oder widersprüchliche Informationen gespeichert oder ausgegeben werden. Diese Validierungsmechanismen gewährleisten die Integrität der gesamten Datenbasis innerhalb des Systems.
## Fehlerbehandlung
Tritt während der Verarbeitung einer Anfrage ein Fehler auf, wird dieser von **L-1** erkannt und entsprechend behandelt. Ungültige oder fehlerhafte Anfragen werden nicht verarbeitet, sondern mit einer passenden Fehlermeldung beantwortet.
Dabei wird sichergestellt, dass keine inkonsistenten Zustände in der Datenbank entstehen. Fehler werden zudem intern erfasst, um eine spätere Analyse und Optimierung des Systems zu ermöglichen.
## Module
**L-1** ist in mehrere Module unterteilt, die jeweils spezifische Aufgaben innerhalb der Datenverarbeitung übernehmen:
- **Service-Manager-Modul** — Koordination der internen Module → [Mehr](./modules/service-manager.md)
- **WebSocket-Modul** — Interne sichere WebSocket-Kommunikation → [Mehr](./modules/websocket-service.md)
- **DB-Service-Modul** — Datenbank-Manager und Pools → [Mehr](./modules/database-service.md)
- **Auth-Modul** — Anfrageauthentifizierung → [Mehr](./modules/auth-service.md)
- **Validierungsmodul** — Datenstruktur- und Konsistenzprüfung → [Mehr](./modules/validate-service.md)
- **Datenverarbeitungsmodul** — Datenbank-Operations-Manager → [Mehr](./modules/operations-manager.md)
- **Loggingmodul** — Systemereignisse und Fehler protokollieren → [Mehr](./modules/logging-service.md)