maddo/Regalamiunsorriso
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages 1 Wiki Activity Actions

feat: add processor service with Redis-backed job queue

Browse source 
- Introduced a new `processor` service in the Docker Compose setup to handle face matching jobs.
- Configured Redis as a job queue and state management system for processing searches.
- Updated the backend to enqueue jobs and manage user locks using Redis.
- Added environment variables for Redis configuration and runtime paths.
- Created technical design documentation for the processor service outlining architecture, queue model, and search lifecycle.
- Updated package.json and package-lock.json to include dependencies for BullMQ and ioredis in the processor workspace.
- Added sample PKL files for local testing in the `test_pkl` directory.
This commit is contained in:
MaddoScientisto 2026-04-11 17:53:22 +02:00
commit 81a1ac85af
20 changed files with 1313 additions and 108 deletions
Download patch file Download diff file Expand all files Collapse all files

127
faceai/README.md
View file

@ -81,6 +81,133 @@ If you change frontend code and want Docker to serve the updated UI, rebuild fir
npm run build
```
## Production Deployment From Registry
The published container is the user-facing FaceAI site only. It already contains:
- the Node/Express backend
- the built Vue frontend assets served by that backend
It does not include:
- the legacy PHP simulator
- the existing `www` site
- the future queue/processor worker
In production, deploy a single FaceAI container behind HTTPS on its own host name, for example `faceai.regalamiunsorriso.it`, and keep the legacy site on its existing stack.
### What The Production Container Exposes
- HTTP service on port `3001` inside the container
- health endpoint at `/health`
- frontend and API from the same process
The image should be run with a reverse proxy or ingress that terminates TLS and forwards traffic to the container.
### Required Runtime Configuration
Set these environment variables for production:
| Variable | Required | Example | Purpose |
| --- | --- | --- | --- |
| `NODE_ENV` | yes | `production` | disables development defaults |
| `PORT` | optional | `3001` | internal listen port |
| `FACEAI_FRONTEND_URL` | yes | `https://faceai.regalamiunsorriso.it` | URL used when the legacy bridge redirects into the app |
| `FACEAI_PUBLIC_BASE_URL` | yes | `https://faceai.regalamiunsorriso.it` | public base URL used for local links and return flow generation |
| `FACEAI_LEGACY_RETURN_URL` | yes | `https://www.regalamiunsorriso.it/faceai_return.php` | legacy endpoint that receives the signed FaceAI result handoff |
| `FACEAI_SHARED_SECRET` | yes | long random secret | shared signing secret between FaceAI and the legacy handoff/return bridge |
| `FACEAI_SESSION_COOKIE` | optional | `rus_faceai_session` | cookie name for the FaceAI session |
| `FACEAI_ENABLE_LOCAL_LEGACY_STATIC` | recommended | `0` | disables development-only static serving of local legacy assets |
Do not enable `FACEAI_ENABLE_LOCAL_LEGACY_STATIC` in production. That mode exists only for local simulator flows.
### Legacy-Side Configuration That Must Match
The container will not work correctly in production unless the legacy bridge is configured consistently.
The legacy site must:
- redirect users into `FACEAI_FRONTEND_URL` with a valid signed handoff token
- use the same `FACEAI_SHARED_SECRET` as the FaceAI container
- expose the configured `FACEAI_LEGACY_RETURN_URL`
- validate the signed return token and fetch the result payload from FaceAI
The shared secret is the trust boundary between the legacy site and FaceAI. Treat it like any other production secret and inject it through the platform secret store, not through source control.
### Example Docker Compose For Production
Replace the registry path and secret values with the real ones from Forgejo.
```yaml
services:
faceai:
image: registry.example.com/my-namespace/faceai:latest
container_name: regalami-faceai
restart: unless-stopped
environment:
NODE_ENV: production
PORT: 3001
FACEAI_FRONTEND_URL: https://faceai.regalamiunsorriso.it
FACEAI_PUBLIC_BASE_URL: https://faceai.regalamiunsorriso.it
FACEAI_LEGACY_RETURN_URL: https://www.regalamiunsorriso.it/faceai_return.php
FACEAI_SHARED_SECRET: change-this-to-a-long-random-secret
FACEAI_SESSION_COOKIE: rus_faceai_session
FACEAI_ENABLE_LOCAL_LEGACY_STATIC: 0
ports:
- "127.0.0.1:3001:3001"
```
This pattern assumes a reverse proxy on the host publishes `https://faceai.regalamiunsorriso.it` and forwards to `127.0.0.1:3001`.
### Example Docker Run
```bash
docker run -d \
--name regalami-faceai \
--restart unless-stopped \
-p 127.0.0.1:3001:3001 \
-e NODE_ENV=production \
-e PORT=3001 \
-e FACEAI_FRONTEND_URL=https://faceai.regalamiunsorriso.it \
-e FACEAI_PUBLIC_BASE_URL=https://faceai.regalamiunsorriso.it \
-e FACEAI_LEGACY_RETURN_URL=https://www.regalamiunsorriso.it/faceai_return.php \
-e FACEAI_SHARED_SECRET=change-this-to-a-long-random-secret \
-e FACEAI_SESSION_COOKIE=rus_faceai_session \
-e FACEAI_ENABLE_LOCAL_LEGACY_STATIC=0 \
registry.example.com/my-namespace/faceai:latest
```
### Reverse Proxy Expectations
The app should sit behind HTTPS. In practice that means:
- publish only the public FaceAI host name externally
- forward the original host and scheme headers from the proxy
- keep the container bound to localhost or a private network if possible
- allow normal browser redirects between the legacy site and the FaceAI host
### Post-Deploy Validation
After the container is up, validate at least the following:
1. `GET /health` returns `{"ok":true}` through the public FaceAI host.
2. The legacy handoff endpoint redirects to `https://faceai.../auth/callback?token=...`.
3. FaceAI can exchange the token and establish a session.
4. Completing a search produces a redirect URL that points to `FACEAI_LEGACY_RETURN_URL`.
5. The legacy return endpoint can resolve the signed result and render the filtered race page.
### Current Production Limitations
This image can be published and deployed, but the current scaffold still has important limitations:
- sessions and search results are stored only in memory, so container restarts lose state
- there is no real queue or processor yet
- there is no persistent storage layer yet
- the backend currently sets the FaceAI session cookie with `secure: false`, which should be hardened before final public rollout
- the local simulator endpoints under `/dev/*` are still present in the app and should be treated as non-production scaffolding
So the registry deployment is appropriate for early hosted integration and controlled production-like rollout, but not yet for the final hardened architecture described in the integration plan
## Environment
Defaults are already set for local development, but these can be overridden:

5
faceai/apps/backend/package.json
View file

@ -8,8 +8,11 @@
"start": "node src/server.js"
},
"dependencies": {
"bullmq": "^5.48.1",
"cookie-parser": "^1.4.7",
"cors": "^2.8.5",
"express": "^4.21.2"
"express": "^4.21.2",
"ioredis": "^5.4.1",
"multer": "^2.0.0"
}
}

10
faceai/apps/backend/src/config.js
View file

@ -14,5 +14,13 @@ export const config = {
: process.env.NODE_ENV !== 'production',
localLegacyStaticRoot: process.env.FACEAI_LOCAL_LEGACY_STATIC_ROOT || defaultLocalLegacyRoot,
sharedSecret: process.env.FACEAI_SHARED_SECRET || 'change-me',
sessionCookieName: process.env.FACEAI_SESSION_COOKIE || 'rus_faceai_session'
sessionCookieName: process.env.FACEAI_SESSION_COOKIE || 'rus_faceai_session',
redisUrl: process.env.FACEAI_REDIS_URL || 'redis://redis:6379',
queueName: process.env.FACEAI_QUEUE_NAME || 'faceai-searches',
runtimeRoot: process.env.FACEAI_RUNTIME_ROOT || '/data/runtime',
uploadRoot: process.env.FACEAI_UPLOAD_ROOT || path.join(process.env.FACEAI_RUNTIME_ROOT || '/data/runtime', 'uploads'),
searchTtlSeconds: Number(process.env.FACEAI_SEARCH_TTL_SECONDS || 24 * 60 * 60),
resultTtlSeconds: Number(process.env.FACEAI_RESULT_TTL_SECONDS || 24 * 60 * 60),
rateLimitWindowSeconds: Number(process.env.FACEAI_RATE_LIMIT_WINDOW_SECONDS || 10 * 60),
rateLimitMaxRequests: Number(process.env.FACEAI_RATE_LIMIT_MAX_REQUESTS || 5)
};

9
faceai/apps/backend/src/matcher-results.js Normal file
View file

@ -0,0 +1,9 @@
export function normalizeMatches(result) {
return (result.matches || []).map((match) => ({
id: match.photoId,
label: match.label || match.photoId,
checkpoint: match.checkpoint || '-',
thumb: match.thumb || match.photoId,
score: match.score ?? null
}));
}

11
faceai/apps/backend/src/queue.js Normal file
View file

@ -0,0 +1,11 @@
import { Queue } from 'bullmq';
let queue = null;
export function getSearchQueue({ queueName, connection }) {
if (!queue) {
queue = new Queue(queueName, { connection });
}
return queue;
}

138
faceai/apps/backend/src/redis-store.js Normal file
View file

@ -0,0 +1,138 @@
import Redis from 'ioredis';
import { randomId } from './auth.js';
export function createRedisConnection(redisUrl) {
return new Redis(redisUrl, {
maxRetriesPerRequest: null,
enableReadyCheck: true
});
}
function searchKey(searchId) {
return `faceai:search:${searchId}`;
}
function resultKey(resultId) {
return `faceai:result:${resultId}`;
}
function activeSearchKey(userId) {
return `faceai:active-search:user:${userId}`;
}
function rateLimitKey(userId) {
return `faceai:rate-limit:${userId}`;
}
export async function incrementRateLimit(redis, userId, windowSeconds) {
const key = rateLimitKey(userId);
const count = await redis.incr(key);
if (count === 1) {
await redis.expire(key, windowSeconds);
}
return count;
}
export async function acquireActiveSearchLock(redis, userId, searchId, ttlSeconds) {
const result = await redis.set(activeSearchKey(userId), searchId, 'EX', ttlSeconds, 'NX');
return result === 'OK';
}
export async function releaseActiveSearchLock(redis, userId, searchId) {
const key = activeSearchKey(userId);
const current = await redis.get(key);
if (current === String(searchId)) {
await redis.del(key);
}
}
export async function getActiveSearchId(redis, userId) {
return redis.get(activeSearchKey(userId));
}
export async function createSearchRecord(redis, payload, ttlSeconds) {
const searchId = randomId('search');
const record = {
id: searchId,
status: 'queued',
resultId: null,
matchCount: 0,
errorCode: null,
errorMessage: null,
createdAt: Date.now(),
startedAt: null,
completedAt: null,
...payload
};
await redis.set(searchKey(searchId), JSON.stringify(record), 'EX', ttlSeconds);
return record;
}
export async function saveSearchRecord(redis, record, ttlSeconds) {
await redis.set(searchKey(record.id), JSON.stringify(record), 'EX', ttlSeconds);
return record;
}
export async function getSearchRecord(redis, searchId) {
const raw = await redis.get(searchKey(searchId));
return raw ? JSON.parse(raw) : null;
}
async function updateSearchRecord(redis, searchId, updater, ttlSeconds) {
const current = await getSearchRecord(redis, searchId);
if (!current) {
return null;
}
const next = updater(current);
await redis.set(searchKey(searchId), JSON.stringify(next), 'EX', ttlSeconds);
return next;
}
export async function markSearchProcessing(redis, searchId, ttlSeconds = 24 * 60 * 60) {
return updateSearchRecord(redis, searchId, (current) => ({
...current,
status: 'processing',
startedAt: Date.now(),
errorCode: null,
errorMessage: null
}), ttlSeconds);
}
export async function markSearchCompleted(redis, searchId, resultId, matchCount, ttlSeconds) {
return updateSearchRecord(redis, searchId, (current) => ({
...current,
status: 'completed',
resultId,
matchCount,
completedAt: Date.now()
}), ttlSeconds);
}
export async function markSearchFailed(redis, searchId, errorCode, errorMessage, ttlSeconds) {
return updateSearchRecord(redis, searchId, (current) => ({
...current,
status: 'failed',
errorCode,
errorMessage,
completedAt: Date.now()
}), ttlSeconds);
}
export async function storeResultRecord(redis, payload, ttlSeconds) {
const resultId = randomId('result');
const record = {
id: resultId,
createdAt: Date.now(),
...payload
};
await redis.set(resultKey(resultId), JSON.stringify(record), 'EX', ttlSeconds);
return record;
}
export async function getResultRecord(redis, resultId) {
const raw = await redis.get(resultKey(resultId));
return raw ? JSON.parse(raw) : null;
}

184
faceai/apps/backend/src/server.js
View file

@ -1,16 +1,49 @@
import express from 'express';
import cors from 'cors';
import cookieParser from 'cookie-parser';
import multer from 'multer';
import fs from 'node:fs';
import fsp from 'node:fs/promises';
import path from 'node:path';
import { fileURLToPath } from 'node:url';
import { config } from './config.js';
import { signPayload, verifySignedPayload } from './auth.js';
import { createSession, createSearch, completeSearch, getResult, getSearch, getSession, mockCatalog } from './store.js';
import { createSession, getSession, mockCatalog } from './store.js';
import {
acquireActiveSearchLock,
createRedisConnection,
createSearchRecord,
getActiveSearchId,
getResultRecord,
getSearchRecord,
incrementRateLimit,
saveSearchRecord
} from './redis-store.js';
import { getSearchQueue } from './queue.js';
import { normalizeMatches } from './matcher-results.js';
const __dirname = path.dirname(fileURLToPath(import.meta.url));
const frontendDist = path.resolve(__dirname, '../../frontend/dist');
const app = express();
const redis = createRedisConnection(config.redisUrl);
const searchQueue = getSearchQueue({ queueName: config.queueName, connection: redis });
await fsp.mkdir(config.uploadRoot, { recursive: true });
const upload = multer({
storage: multer.diskStorage({
destination: (req, file, cb) => {
const pendingRoot = path.join(config.uploadRoot, 'pending');
fsp.mkdir(pendingRoot, { recursive: true })
.then(() => cb(null, pendingRoot))
.catch((error) => cb(error));
},
filename: (req, file, cb) => {
const safeName = file.originalname.replace(/[^a-zA-Z0-9._-]/g, '_');
cb(null, `${Date.now()}_${safeName}`);
}
})
});
app.use(cookieParser());
app.use(express.json());
@ -42,6 +75,25 @@ function requireSession(req, res, next) {
next();
}
async function enforceSearchRateLimit(req, res, next) {
const userId = req.faceaiSession?.user?.id;
if (!userId) {
res.status(401).json({ error: 'Not authenticated with FaceAI' });
return;
}
const count = await incrementRateLimit(redis, userId, config.rateLimitWindowSeconds);
if (count > config.rateLimitMaxRequests) {
res.status(429).json({
error: 'Too many search attempts. Please try again later.',
code: 'RATE_LIMITED'
});
return;
}
next();
}
function issueHandoffToken({ raceId, raceSlug, lang, returnUrl }) {
const race = mockCatalog[raceId] || { id: raceId, slug: raceSlug || `race-${raceId}`, name: raceSlug || `Race ${raceId}` };
@ -185,7 +237,7 @@ app.get('/dev/legacy/launch', (req, res) => {
res.redirect(`${config.frontendUrl}/auth/callback?token=${encodeURIComponent(token)}`);
});
app.get('/dev/legacy/return', (req, res) => {
app.get('/dev/legacy/return', async (req, res) => {
try {
const token = String(req.query.token || '');
const payload = verifySignedPayload(token, config.sharedSecret);
@ -193,12 +245,17 @@ app.get('/dev/legacy/return', (req, res) => {
throw new Error('Wrong token type');
}
const result = getResult(String(req.query.resultId || payload.resultId));
const result = await getResultRecord(redis, String(req.query.resultId || payload.resultId));
if (!result || result.userId !== payload.userId) {
throw new Error('Result not found');
}
res.type('html').send(renderLegacyRacePage({ raceId: result.raceId, lang: result.lang || 'it', result }));
const normalizedResult = {
...result,
matches: normalizeMatches(result)
};
res.type('html').send(renderLegacyRacePage({ raceId: result.raceId, lang: result.lang || 'it', result: normalizedResult }));
} catch (error) {
res.status(400).type('html').send(`<h1>Return handoff failed</h1><p>${escapeHtml(error.message)}</p>`);
}
@ -247,33 +304,86 @@ app.get('/api/session', requireSession, (req, res) => {
res.json(req.faceaiSession);
});
app.post('/api/searches', requireSession, (req, res) => {
app.post('/api/searches', requireSession, enforceSearchRateLimit, upload.single('selfie'), async (req, res) => {
try {
const raceId = String(req.body.raceId || req.faceaiSession.race.id);
const selfieName = String(req.body.selfieName || 'selfie.jpg');
const userId = String(req.faceaiSession.user.id);
const activeSearchId = await getActiveSearchId(redis, userId);
const search = createSearch({
raceId,
selfieName,
user: req.faceaiSession.user,
returnUrl: req.faceaiSession.returnUrl,
lang: req.faceaiSession.lang
if (activeSearchId) {
res.status(409).json({
error: 'There is already an operation being processed.',
code: 'ACTIVE_SEARCH_EXISTS',
activeSearchId
});
return;
}
setTimeout(() => {
completeSearch(search.id);
}, 3500);
if (!req.file) {
res.status(400).json({
error: 'Choose a selfie before starting the search.',
code: 'MISSING_SELFIE'
});
return;
}
const race = mockCatalog[raceId] || req.faceaiSession.race;
const search = await createSearchRecord(redis, {
raceId,
raceName: race?.name || raceId,
userId,
returnUrl: req.faceaiSession.returnUrl,
lang: req.faceaiSession.lang,
selfieName: req.file.originalname,
selfiePath: req.file.path,
uploadPath: req.file.path
}, config.searchTtlSeconds);
const lockAcquired = await acquireActiveSearchLock(redis, userId, search.id, config.searchTtlSeconds);
if (!lockAcquired) {
await fsp.unlink(req.file.path).catch(() => {});
res.status(409).json({
error: 'There is already an operation being processed.',
code: 'ACTIVE_SEARCH_EXISTS'
});
return;
}
const finalUploadDir = path.join(config.uploadRoot, search.id);
await fsp.mkdir(finalUploadDir, { recursive: true });
const finalUploadPath = path.join(finalUploadDir, path.basename(req.file.path));
await fsp.rename(req.file.path, finalUploadPath);
const updatedSearch = await saveSearchRecord(redis, {
...search,
selfiePath: finalUploadPath,
uploadPath: finalUploadPath
}, config.searchTtlSeconds);
await searchQueue.add('run-search', {
searchId: search.id
}, {
removeOnComplete: 100,
removeOnFail: 100
});
res.status(201).json({
id: search.id,
status: search.status,
raceId: search.raceId,
selfieName: search.selfieName
id: updatedSearch.id,
status: updatedSearch.status,
raceId: updatedSearch.raceId,
selfieName: updatedSearch.selfieName,
matchCount: updatedSearch.matchCount,
errorCode: updatedSearch.errorCode,
errorMessage: updatedSearch.errorMessage
});
} catch (error) {
res.status(500).json({ error: error.message || 'Unable to create the search.' });
}
});
app.get('/api/searches/:id', requireSession, (req, res) => {
const search = getSearch(req.params.id);
if (!search || search.user.id !== req.faceaiSession.user.id) {
app.get('/api/searches/:id', requireSession, async (req, res) => {
const search = await getSearchRecord(redis, req.params.id);
if (!search || search.userId !== req.faceaiSession.user.id) {
res.status(404).json({ error: 'Search not found' });
return;
}
@ -285,13 +395,15 @@ app.get('/api/searches/:id', requireSession, (req, res) => {
resultId: search.resultId,
createdAt: search.createdAt,
completedAt: search.completedAt,
matchCount: search.matches.length
matchCount: search.matchCount || 0,
errorCode: search.errorCode,
errorMessage: search.errorMessage
});
});
app.get('/api/searches/:id/redirect', requireSession, (req, res) => {
const search = getSearch(req.params.id);
if (!search || search.user.id !== req.faceaiSession.user.id) {
app.get('/api/searches/:id/redirect', requireSession, async (req, res) => {
const search = await getSearchRecord(redis, req.params.id);
if (!search || search.userId !== req.faceaiSession.user.id) {
res.status(404).json({ error: 'Search not found' });
return;
}
@ -301,7 +413,12 @@ app.get('/api/searches/:id/redirect', requireSession, (req, res) => {
return;
}
const result = getResult(search.resultId);
const result = await getResultRecord(redis, search.resultId);
if (!result) {
res.status(404).json({ error: 'Result not found' });
return;
}
const token = issueReturnToken(result);
res.json({
@ -309,7 +426,7 @@ app.get('/api/searches/:id/redirect', requireSession, (req, res) => {
});
});
app.get('/bridge/results/:id', (req, res) => {
app.get('/bridge/results/:id', async (req, res) => {
try {
const token = String(req.query.token || '');
const payload = verifySignedPayload(token, config.sharedSecret);
@ -321,7 +438,7 @@ app.get('/bridge/results/:id', (req, res) => {
throw new Error('Result id mismatch');
}
const result = getResult(req.params.id);
const result = await getResultRecord(redis, req.params.id);
if (!result || result.userId !== payload.userId) {
throw new Error('Result not found');
}
@ -340,6 +457,15 @@ app.get('/bridge/results/:id', (req, res) => {
}
});
app.get('/api/health/queue', async (req, res) => {
try {
await redis.ping();
res.json({ ok: true });
} catch (error) {
res.status(500).json({ ok: false, error: error.message });
}
});
if (fs.existsSync(frontendDist)) {
app.use(express.static(frontendDist));
app.get('*', (req, res, next) => {

61
faceai/apps/backend/src/store.js
View file

@ -34,8 +34,6 @@ export const mockCatalog = {
};
const sessions = new Map();
const searches = new Map();
const results = new Map();
export function createSession(session) {
const sessionId = randomId('sess');
@ -49,62 +47,3 @@ export function createSession(session) {
export function getSession(sessionId) {
return sessions.get(sessionId) || null;
}
export function createSearch({ raceId, user, selfieName, returnUrl, lang }) {
const searchId = randomId('search');
searches.set(searchId, {
id: searchId,
raceId,
user,
selfieName,
returnUrl,
lang,
status: 'processing',
createdAt: Date.now(),
completedAt: null,
resultId: null,
matches: []
});
return searches.get(searchId);
}
export function getSearch(searchId) {
return searches.get(searchId) || null;
}
export function completeSearch(searchId) {
const search = searches.get(searchId);
if (!search) {
return null;
}
const race = mockCatalog[search.raceId];
const matches = (race?.photos || []).slice(0, Math.min(4, race?.photos?.length || 0));
const resultId = randomId('result');
results.set(resultId, {
id: resultId,
raceId: search.raceId,
raceName: race?.name || search.raceId,
userId: search.user.id,
returnUrl: search.returnUrl,
lang: search.lang,
matches,
createdAt: Date.now()
});
const completed = {
...search,
status: 'completed',
completedAt: Date.now(),
resultId,
matches
};
searches.set(searchId, completed);
return completed;
}
export function getResult(resultId) {
return results.get(resultId) || null;
}

22
faceai/apps/frontend/src/views/HomeView.vue
View file

@ -46,6 +46,10 @@ const statusLabel = computed(() => {
return `Ricerca completata. Trovate ${activeSearch.value.matchCount} foto corrispondenti.`;
}
if (activeSearch.value.status === 'failed') {
return 'La ricerca non e stata completata. Verifica il messaggio di errore e riprova.';
}
return 'Ricerca in corso. Il sistema aggiorna automaticamente lo stato finche il risultato non e pronto.';
});
@ -75,6 +79,12 @@ async function pollSearch(searchId) {
}
activeSearch.value = await response.json();
if (activeSearch.value.status === 'failed') {
isSubmitting.value = false;
errorMessage.value = activeSearch.value.errorMessage || 'The search failed.';
return;
}
if (activeSearch.value.status === 'completed') {
isSubmitting.value = false;
const redirectResponse = await fetch(`/api/searches/${searchId}/redirect`, { credentials: 'include' });
@ -108,16 +118,14 @@ async function submitSearch() {
isSubmitting.value = true;
const formData = new FormData();
formData.set('raceId', session.value.race.id);
formData.set('selfie', selectedFile.value);
const response = await fetch('/api/searches', {
method: 'POST',
headers: {
'Content-Type': 'application/json'
},
credentials: 'include',
body: JSON.stringify({
raceId: session.value.race.id,
selfieName: selectedFile.value.name
})
body: formData
});
const payload = await response.json();

14
faceai/apps/processor/package.json Normal file
View file

@ -0,0 +1,14 @@
{
"name": "@regalami/faceai-processor",
"private": true,
"type": "module",
"scripts": {
"dev": "node --watch src/worker.js",
"build": "node -e \"console.log('processor build not required')\"",
"start": "node src/worker.js"
},
"dependencies": {
"bullmq": "^5.48.1",
"ioredis": "^5.4.1"
}
}

12
faceai/apps/processor/src/config.js Normal file
View file

@ -0,0 +1,12 @@
export const config = {
redisUrl: process.env.FACEAI_REDIS_URL || 'redis://redis:6379',
queueName: process.env.FACEAI_QUEUE_NAME || 'faceai-searches',
workerConcurrency: Number(process.env.FACEAI_WORKER_CONCURRENCY || 2),
workerTimeoutMs: Number(process.env.FACEAI_WORKER_TIMEOUT_MS || 5 * 60 * 1000),
runtimeRoot: process.env.FACEAI_RUNTIME_ROOT || '/data/runtime',
pklRoot: process.env.FACEAI_PKL_ROOT || '/data/pkl',
fallbackPklRoot: process.env.FACEAI_TEST_PKL_ROOT || '/data/pkl/test',
matcherBinary: process.env.FACEAI_MATCHER_BINARY || '/opt/face-recognition/face_matcher',
searchTtlSeconds: Number(process.env.FACEAI_SEARCH_TTL_SECONDS || 24 * 60 * 60),
resultTtlSeconds: Number(process.env.FACEAI_RESULT_TTL_SECONDS || 24 * 60 * 60)
};

99
faceai/apps/processor/src/worker-utils.js Normal file
View file

@ -0,0 +1,99 @@
import fs from 'node:fs/promises';
import path from 'node:path';
import { spawn } from 'node:child_process';
async function fileExists(filePath) {
try {
await fs.access(filePath);
return true;
} catch {
return false;
}
}
export async function resolvePklPath({ raceId, pklRoot, fallbackPklRoot }) {
const preferred = path.join(pklRoot, String(raceId), 'face_encodings.pkl');
if (await fileExists(preferred)) {
return preferred;
}
const flatFile = path.join(pklRoot, `${raceId}.pkl`);
if (await fileExists(flatFile)) {
return flatFile;
}
const fallbackEntries = await fs.readdir(fallbackPklRoot).catch(() => []);
const fallbackFile = fallbackEntries.find((entry) => entry.toLowerCase().endsWith('.pkl'));
if (fallbackFile) {
return path.join(fallbackPklRoot, fallbackFile);
}
throw new Error(`No PKL file available for race ${raceId}`);
}
export async function runFaceMatcher({ matcherBinary, selfiePath, pklPath, csvPath, logPath, timeoutMs }) {
await fs.mkdir(path.dirname(csvPath), { recursive: true });
await fs.mkdir(path.dirname(logPath), { recursive: true });
return new Promise((resolve, reject) => {
const child = spawn(matcherBinary, [
'--image', selfiePath,
'--encodings', pklPath,
'--out', csvPath,
'--log', logPath
], {
stdio: 'ignore'
});
const timer = setTimeout(() => {
child.kill('SIGKILL');
reject(new Error('face_matcher timed out'));
}, timeoutMs);
child.on('error', (error) => {
clearTimeout(timer);
reject(error);
});
child.on('exit', (code) => {
clearTimeout(timer);
if (code === 0) {
resolve();
return;
}
reject(new Error(`face_matcher exited with code ${code}`));
});
});
}
export async function parseMatcherCsv(csvPath) {
const content = await fs.readFile(csvPath, 'utf8');
const lines = content
.split(/\r?\n/)
.map((line) => line.trim())
.filter(Boolean);
if (!lines.length) {
return [];
}
const rows = lines.map((line) => line.split(',').map((part) => part.trim().replace(/^"|"$/g, '')));
const firstRow = rows[0];
const hasHeader = firstRow.some((cell) => /file|image|score|distance|confidence/i.test(cell));
const dataRows = hasHeader ? rows.slice(1) : rows;
return dataRows
.filter((cells) => cells[0])
.map((cells) => {
const photoId = path.basename(cells[0]);
const numericCell = cells.find((cell, index) => index > 0 && !Number.isNaN(Number(cell)));
const score = numericCell ? Number(numericCell) : null;
return {
photoId,
score,
label: photoId
};
});
}

82
faceai/apps/processor/src/worker.js Normal file
View file

@ -0,0 +1,82 @@
import fs from 'node:fs/promises';
import path from 'node:path';
import { Worker } from 'bullmq';
import { config } from './config.js';
import {
createRedisConnection,
getSearchRecord,
markSearchCompleted,
markSearchFailed,
markSearchProcessing,
releaseActiveSearchLock,
storeResultRecord
} from '../../backend/src/redis-store.js';
import { parseMatcherCsv, resolvePklPath, runFaceMatcher } from './worker-utils.js';
const connection = createRedisConnection(config.redisUrl);
async function processJob(job) {
const searchId = String(job.data.searchId || '');
const search = await getSearchRecord(connection, searchId);
if (!search) {
throw new Error(`Search ${searchId} not found`);
}
await markSearchProcessing(connection, searchId, config.searchTtlSeconds);
const searchDir = path.join(config.runtimeRoot, 'searches', searchId);
await fs.mkdir(searchDir, { recursive: true });
try {
const pklPath = await resolvePklPath({
raceId: search.raceId,
pklRoot: config.pklRoot,
fallbackPklRoot: config.fallbackPklRoot
});
const csvPath = path.join(searchDir, 'result.csv');
const logPath = path.join(searchDir, 'matcher.log');
await runFaceMatcher({
matcherBinary: config.matcherBinary,
selfiePath: search.selfiePath,
pklPath,
csvPath,
logPath,
timeoutMs: config.workerTimeoutMs
});
const matches = await parseMatcherCsv(csvPath);
const result = await storeResultRecord(connection, {
raceId: search.raceId,
raceName: search.raceName,
userId: search.userId,
returnUrl: search.returnUrl,
lang: search.lang,
matches
}, config.resultTtlSeconds);
await markSearchCompleted(connection, searchId, result.id, matches.length, config.searchTtlSeconds);
await releaseActiveSearchLock(connection, search.userId, searchId);
} catch (error) {
await markSearchFailed(connection, searchId, 'PROCESSOR_ERROR', error.message, config.searchTtlSeconds);
await releaseActiveSearchLock(connection, search.userId, searchId);
throw error;
}
}
const worker = new Worker(config.queueName, processJob, {
connection,
concurrency: config.workerConcurrency
});
worker.on('completed', (job) => {
console.log(`Completed FaceAI search ${job.data.searchId}`);
});
worker.on('failed', (job, error) => {
const searchId = job?.data?.searchId || 'unknown';
console.error(`Failed FaceAI search ${searchId}: ${error.message}`);
});
console.log(`FaceAI processor listening on queue ${config.queueName} with concurrency ${config.workerConcurrency}`);

35
faceai/docker-compose.yml
View file

@ -13,11 +13,43 @@ services:
FACEAI_LOCAL_LEGACY_STATIC_ROOT: /legacy-www
FACEAI_SHARED_SECRET: change-me
FACEAI_SESSION_COOKIE: rus_faceai_session
FACEAI_REDIS_URL: redis://redis:6379
FACEAI_RUNTIME_ROOT: /data/runtime
FACEAI_UPLOAD_ROOT: /data/runtime/uploads
volumes:
- .:/app
- ../www:/legacy-www:ro
- faceai-runtime:/data/runtime
ports:
- "3001:3001"
depends_on:
- redis
processor:
image: node:20-bookworm-slim
container_name: regalami-faceai-processor
working_dir: /app
command: sh -c "npm run start --workspace @regalami/faceai-processor"
environment:
FACEAI_REDIS_URL: redis://redis:6379
FACEAI_QUEUE_NAME: faceai-searches
FACEAI_RUNTIME_ROOT: /data/runtime
FACEAI_PKL_ROOT: /data/pkl
FACEAI_TEST_PKL_ROOT: /data/pkl/test
FACEAI_WORKER_CONCURRENCY: 2
FACEAI_MATCHER_BINARY: /opt/face-recognition/face_matcher
volumes:
- .:/app
- ../bin/Face_Recognition_Unix:/opt/face-recognition:ro
- ../test_pkl:/data/pkl/test:ro
- faceai-runtime:/data/runtime
depends_on:
- redis
redis:
image: redis:7-alpine
container_name: regalami-faceai-redis
command: redis-server --appendonly no
legacy-php:
image: php:8.3-apache
@ -32,3 +64,6 @@ services:
- ../www:/var/www/html
ports:
- "8080:80"
volumes:
faceai-runtime:

166
faceai/docs/processor-technical-design.md Normal file
View file

@ -0,0 +1,166 @@
# FaceAI Processor Technical Design
## Goal
Add an internal processor service that executes `face_matcher` jobs for the public FaceAI site, while preventing duplicate searches per user and keeping all state short-lived and restart-safe.
## Scope Of This Slice
- add Redis-backed queue and job state
- add a dedicated `processor` workspace and container scaffold
- replace in-memory search orchestration in the public backend
- preserve the existing frontend polling and legacy return flow
- support local PKL testing from `test_pkl/`
This slice does not yet implement production NAS mounting, persistent databases, or a final parser tailored to the real matcher CSV format.
## Runtime Architecture
### Public backend
- owns the authenticated API used by the Vue frontend
- stores uploaded selfies in a shared runtime volume
- enqueues jobs into BullMQ
- keeps per-search state, results, rate limits, and active-user locks in Redis
- never executes `face_matcher` directly
### Processor
- consumes queue jobs from Redis using BullMQ worker concurrency
- resolves the race-scoped PKL path for each job
- executes the Linux `face_matcher` binary
- parses the CSV result into legacy-compatible `photoId` matches
- writes final state and result payload back to Redis
### Redis
- queue broker for BullMQ
- source of truth for active-user locks
- source of truth for search status and short-lived results
- source of truth for rate-limit counters
## Queue And Locking Model
- queue name: `faceai-searches`
- active lock key: `faceai:active-search:user:{legacyUserId}`
- search record key: `faceai:search:{searchId}`
- result record key: `faceai:result:{resultId}`
- rate limit key prefix: `faceai:rate-limit:{legacyUserId}`
`POST /api/searches` must acquire the active-user lock before enqueueing. If the lock already exists, the backend returns `409` with error code `ACTIVE_SEARCH_EXISTS`.
The lock is released only when the processor marks the search as terminal: `completed`, `failed`, or `timed_out`.
## Race And PKL Resolution
The canonical race key is the legacy `id_gara`, already exposed as `raceId` in the existing handoff flow.
The processor resolves the PKL path using a race-based directory layout:
```text
/data/pkl/
101/
face_encodings.pkl
202/
face_encodings.pkl
```
The lookup rule is:
1. try `/data/pkl/{raceId}/face_encodings.pkl`
2. optionally fall back to `/data/pkl/{raceId}.pkl`
3. fail the job if neither exists
For local development, `test_pkl/` is mounted into `/data/pkl/test` and the backend can fall back to the first `.pkl` file in that folder when no race-specific file exists yet.
## Shared Runtime Storage
Both the public backend and the processor mount the same writable runtime directory:
```text
/data/runtime/
uploads/
searches/
```
- uploaded selfies are written under `uploads/{searchId}/`
- worker output and logs are written under `searches/{searchId}/`
- cleanup can safely remove old per-search directories after retention expires
## Search Lifecycle
1. frontend uploads a selfie and calls `POST /api/searches`
2. backend validates session, rate limit, and active-user lock
3. backend stores the upload and creates a Redis search record with status `queued`
4. backend enqueues a BullMQ job
5. processor picks up the job and sets status `processing`
6. processor runs `face_matcher`
7. processor parses CSV output into matches
8. processor stores a result record and marks the search `completed`
9. frontend polling reads Redis-backed state through `GET /api/searches/:id`
10. existing redirect flow sends the user back to the legacy filtered page
## Search Record Shape
```json
{
"id": "search_...",
"status": "queued",
"raceId": "101",
"userId": "legacy-user-1",
"returnUrl": "https://...",
"lang": "it",
"selfieName": "selfie.jpg",
"selfiePath": "/data/runtime/uploads/search_.../selfie.jpg",
"resultId": null,
"matchCount": 0,
"errorCode": null,
"errorMessage": null,
"createdAt": 0,
"startedAt": null,
"completedAt": null
}
```
## Result Shape
```json
{
"id": "result_...",
"raceId": "101",
"raceName": "Mezza di Firenze",
"userId": "legacy-user-1",
"returnUrl": "https://...",
"lang": "it",
"matches": [
{
"photoId": "legacy-photo-id",
"score": 0.98,
"label": "legacy-photo-id"
}
],
"createdAt": 0
}
```
## Compose Topology
- `faceai`: public backend plus built frontend
- `processor`: queue consumer and matcher executor
- `redis`: queue and short-lived state
- `legacy-php`: local bridge simulator for end-to-end testing
## Operational Defaults
- worker concurrency: `2`
- active search retention: `24h`
- result retention: `24h`
- rate limit window: `5 requests / 10 minutes / user`
- worker timeout: `5 minutes`
## Known Follow-Up Work
- confirm the real CSV columns emitted by `face_matcher`
- verify the Linux binary shared library requirements inside the processor image
- replace the PKL fallback with a strict NAS-backed race mapping once the final folder layout is agreed
- add cleanup jobs for expired runtime files

431
faceai/package-lock.json generated
View file

@ -7,7 +7,8 @@
"name": "faceai",
"workspaces": [
"apps/frontend",
"apps/backend"
"apps/backend",
"apps/processor"
],
"devDependencies": {
"concurrently": "^9.1.2"
@ -16,9 +17,12 @@
"apps/backend": {
"name": "@regalami/faceai-backend",
"dependencies": {
"bullmq": "^5.48.1",
"cookie-parser": "^1.4.7",
"cors": "^2.8.5",
"express": "^4.21.2"
"express": "^4.21.2",
"ioredis": "^5.4.1",
"multer": "^2.0.0"
}
},
"apps/frontend": {
@ -32,6 +36,13 @@
"vite": "^6.1.0"
}
},
"apps/processor": {
"name": "@regalami/faceai-processor",
"dependencies": {
"bullmq": "^5.48.1",
"ioredis": "^5.4.1"
}
},
"node_modules/@babel/helper-string-parser": {
"version": "7.27.1",
"resolved": "https://registry.npmjs.org/@babel/helper-string-parser/-/helper-string-parser-7.27.1.tgz",
@ -520,12 +531,96 @@
"node": ">=18"
}
},
"node_modules/@ioredis/commands": {
"version": "1.5.1",
"resolved": "https://registry.npmjs.org/@ioredis/commands/-/commands-1.5.1.tgz",
"integrity": "sha512-JH8ZL/ywcJyR9MmJ5BNqZllXNZQqQbnVZOqpPQqE1vHiFgAw4NHbvE0FOduNU8IX9babitBT46571OnPTT0Zcw==",
"license": "MIT"
},
"node_modules/@jridgewell/sourcemap-codec": {
"version": "1.5.5",
"resolved": "https://registry.npmjs.org/@jridgewell/sourcemap-codec/-/sourcemap-codec-1.5.5.tgz",
"integrity": "sha512-cYQ9310grqxueWbl+WuIUIaiUaDcj7WOq5fVhEljNVgRfOUhY9fy2zTvfoqWsnebh8Sl70VScFbICvJnLKB0Og==",
"license": "MIT"
},
"node_modules/@msgpackr-extract/msgpackr-extract-darwin-arm64": {
"version": "3.0.3",
"resolved": "https://registry.npmjs.org/@msgpackr-extract/msgpackr-extract-darwin-arm64/-/msgpackr-extract-darwin-arm64-3.0.3.tgz",
"integrity": "sha512-QZHtlVgbAdy2zAqNA9Gu1UpIuI8Xvsd1v8ic6B2pZmeFnFcMWiPLfWXh7TVw4eGEZ/C9TH281KwhVoeQUKbyjw==",
"cpu": [
"arm64"
],
"license": "MIT",
"optional": true,
"os": [
"darwin"
]
},
"node_modules/@msgpackr-extract/msgpackr-extract-darwin-x64": {
"version": "3.0.3",
"resolved": "https://registry.npmjs.org/@msgpackr-extract/msgpackr-extract-darwin-x64/-/msgpackr-extract-darwin-x64-3.0.3.tgz",
"integrity": "sha512-mdzd3AVzYKuUmiWOQ8GNhl64/IoFGol569zNRdkLReh6LRLHOXxU4U8eq0JwaD8iFHdVGqSy4IjFL4reoWCDFw==",
"cpu": [
"x64"
],
"license": "MIT",
"optional": true,
"os": [
"darwin"
]
},
"node_modules/@msgpackr-extract/msgpackr-extract-linux-arm": {
"version": "3.0.3",
"resolved": "https://registry.npmjs.org/@msgpackr-extract/msgpackr-extract-linux-arm/-/msgpackr-extract-linux-arm-3.0.3.tgz",
"integrity": "sha512-fg0uy/dG/nZEXfYilKoRe7yALaNmHoYeIoJuJ7KJ+YyU2bvY8vPv27f7UKhGRpY6euFYqEVhxCFZgAUNQBM3nw==",
"cpu": [
"arm"
],
"license": "MIT",
"optional": true,
"os": [
"linux"
]
},
"node_modules/@msgpackr-extract/msgpackr-extract-linux-arm64": {
"version": "3.0.3",
"resolved": "https://registry.npmjs.org/@msgpackr-extract/msgpackr-extract-linux-arm64/-/msgpackr-extract-linux-arm64-3.0.3.tgz",
"integrity": "sha512-YxQL+ax0XqBJDZiKimS2XQaf+2wDGVa1enVRGzEvLLVFeqa5kx2bWbtcSXgsxjQB7nRqqIGFIcLteF/sHeVtQg==",
"cpu": [
"arm64"
],
"license": "MIT",
"optional": true,
"os": [
"linux"
]
},
"node_modules/@msgpackr-extract/msgpackr-extract-linux-x64": {
"version": "3.0.3",
"resolved": "https://registry.npmjs.org/@msgpackr-extract/msgpackr-extract-linux-x64/-/msgpackr-extract-linux-x64-3.0.3.tgz",
"integrity": "sha512-cvwNfbP07pKUfq1uH+S6KJ7dT9K8WOE4ZiAcsrSes+UY55E/0jLYc+vq+DO7jlmqRb5zAggExKm0H7O/CBaesg==",
"cpu": [
"x64"
],
"license": "MIT",
"optional": true,
"os": [
"linux"
]
},
"node_modules/@msgpackr-extract/msgpackr-extract-win32-x64": {
"version": "3.0.3",
"resolved": "https://registry.npmjs.org/@msgpackr-extract/msgpackr-extract-win32-x64/-/msgpackr-extract-win32-x64-3.0.3.tgz",
"integrity": "sha512-x0fWaQtYp4E6sktbsdAqnehxDgEc/VwM7uLsRCYWaiGu0ykYdZPiS8zCWdnjHwyiumousxfBm4SO31eXqwEZhQ==",
"cpu": [
"x64"
],
"license": "MIT",
"optional": true,
"os": [
"win32"
]
},
"node_modules/@regalami/faceai-backend": {
"resolved": "apps/backend",
"link": true
@ -534,6 +629,10 @@
"resolved": "apps/frontend",
"link": true
},
"node_modules/@regalami/faceai-processor": {
"resolved": "apps/processor",
"link": true
},
"node_modules/@rollup/rollup-android-arm-eabi": {
"version": "4.60.1",
"resolved": "https://registry.npmjs.org/@rollup/rollup-android-arm-eabi/-/rollup-android-arm-eabi-4.60.1.tgz",
@ -1050,6 +1149,12 @@
"url": "https://github.com/chalk/ansi-styles?sponsor=1"
}
},
"node_modules/append-field": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/append-field/-/append-field-1.0.0.tgz",
"integrity": "sha512-klpgFSWLW1ZEs8svjfb7g4qWY0YS5imI82dTg+QahUvJ8YqAY0P10Uk8tTyh9ZGuYEZEMaeJYCF5BFuX552hsw==",
"license": "MIT"
},
"node_modules/array-flatten": {
"version": "1.1.1",
"resolved": "https://registry.npmjs.org/array-flatten/-/array-flatten-1.1.1.tgz",
@ -1080,6 +1185,38 @@
"npm": "1.2.8000 || >= 1.4.16"
}
},
"node_modules/buffer-from": {
"version": "1.1.2",
"resolved": "https://registry.npmjs.org/buffer-from/-/buffer-from-1.1.2.tgz",
"integrity": "sha512-E+XQCRwSbaaiChtv6k6Dwgc+bx+Bs6vuKJHHl5kox/BaKbhiXzqQOwK4cO22yElGp2OCmjwVhT3HmxgyPGnJfQ==",
"license": "MIT"
},
"node_modules/bullmq": {
"version": "5.73.4",
"resolved": "https://registry.npmjs.org/bullmq/-/bullmq-5.73.4.tgz",
"integrity": "sha512-Q+NeFLtdKSD3GDPYSX4pH+Mc9E4OZVKimXwrnZ5WmndNy31COMy4vQV9zfhgfHGSUFrlpsBicfKYbSjx9FbO+A==",
"license": "MIT",
"dependencies": {
"cron-parser": "4.9.0",
"ioredis": "5.10.1",
"msgpackr": "1.11.5",
"node-abort-controller": "3.1.1",
"semver": "7.7.4",
"tslib": "2.8.1",
"uuid": "11.1.0"
}
},
"node_modules/busboy": {
"version": "1.6.0",
"resolved": "https://registry.npmjs.org/busboy/-/busboy-1.6.0.tgz",
"integrity": "sha512-8SFQbg/0hQ9xy3UNTB0YEnsNBbWfhf7RtnzpL7TkBiTBRfrQ9Fxcnz7VJsleJpyp6rVLvXiuORqjlHi5q+PYuA==",
"dependencies": {
"streamsearch": "^1.1.0"
},
"engines": {
"node": ">=10.16.0"
}
},
"node_modules/bytes": {
"version": "3.1.2",
"resolved": "https://registry.npmjs.org/bytes/-/bytes-3.1.2.tgz",
@ -1163,6 +1300,15 @@
"node": ">=12"
}
},
"node_modules/cluster-key-slot": {
"version": "1.1.2",
"resolved": "https://registry.npmjs.org/cluster-key-slot/-/cluster-key-slot-1.1.2.tgz",
"integrity": "sha512-RMr0FhtfXemyinomL4hrWcYJxmX6deFdCxpJzhDttxgO1+bcCnkk+9drydLVDmAMG7NE6aN/fl4F7ucU/90gAA==",
"license": "Apache-2.0",
"engines": {
"node": ">=0.10.0"
}
},
"node_modules/color-convert": {
"version": "2.0.1",
"resolved": "https://registry.npmjs.org/color-convert/-/color-convert-2.0.1.tgz",
@ -1183,6 +1329,21 @@
"dev": true,
"license": "MIT"
},
"node_modules/concat-stream": {
"version": "2.0.0",
"resolved": "https://registry.npmjs.org/concat-stream/-/concat-stream-2.0.0.tgz",
"integrity": "sha512-MWufYdFw53ccGjCA+Ol7XJYpAlW6/prSMzuPOTRnJGcGzuhLn4Scrz7qf6o8bROZ514ltazcIFJZevcfbo0x7A==",
"engines": [
"node >= 6.0"
],
"license": "MIT",
"dependencies": {
"buffer-from": "^1.0.0",
"inherits": "^2.0.3",
"readable-stream": "^3.0.2",
"typedarray": "^0.0.6"
}
},
"node_modules/concurrently": {
"version": "9.2.1",
"resolved": "https://registry.npmjs.org/concurrently/-/concurrently-9.2.1.tgz",
@ -1274,6 +1435,18 @@
"url": "https://opencollective.com/express"
}
},
"node_modules/cron-parser": {
"version": "4.9.0",
"resolved": "https://registry.npmjs.org/cron-parser/-/cron-parser-4.9.0.tgz",
"integrity": "sha512-p0SaNjrHOnQeR8/VnfGbmg9te2kfyYSQ7Sc/j/6DtPL3JQvKxmjO9TSjNFpujqV3vEYYBvNNvXSxzyksBWAx1Q==",
"license": "MIT",
"dependencies": {
"luxon": "^3.2.1"
},
"engines": {
"node": ">=12.0.0"
}
},
"node_modules/csstype": {
"version": "3.2.3",
"resolved": "https://registry.npmjs.org/csstype/-/csstype-3.2.3.tgz",
@ -1289,6 +1462,15 @@
"ms": "2.0.0"
}
},
"node_modules/denque": {
"version": "2.1.0",
"resolved": "https://registry.npmjs.org/denque/-/denque-2.1.0.tgz",
"integrity": "sha512-HVQE3AAb/pxF8fQAoiqpvg9i3evqug3hoiwakOyZAwJm+6vZehbkYXZ0l4JxS+I3QxM97v5aaRNhj8v5oBhekw==",
"license": "Apache-2.0",
"engines": {
"node": ">=0.10"
}
},
"node_modules/depd": {
"version": "2.0.0",
"resolved": "https://registry.npmjs.org/depd/-/depd-2.0.0.tgz",
@ -1308,6 +1490,16 @@
"npm": "1.2.8000 || >= 1.4.16"
}
},
"node_modules/detect-libc": {
"version": "2.1.2",
"resolved": "https://registry.npmjs.org/detect-libc/-/detect-libc-2.1.2.tgz",
"integrity": "sha512-Btj2BOOO83o3WyH59e8MgXsxEQVcarkUOpEYrubB0urwnN10yQ364rsiByU11nZlqWYZm05i/of7io4mzihBtQ==",
"license": "Apache-2.0",
"optional": true,
"engines": {
"node": ">=8"
}
},
"node_modules/dunder-proto": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/dunder-proto/-/dunder-proto-1.0.1.tgz",
@ -1714,6 +1906,53 @@
"integrity": "sha512-k/vGaX4/Yla3WzyMCvTQOXYeIHvqOKtnqBduzTHpzpQZzAskKMhZ2K+EnBiSM9zGSoIFeMpXKxa4dYeZIQqewQ==",
"license": "ISC"
},
"node_modules/ioredis": {
"version": "5.10.1",
"resolved": "https://registry.npmjs.org/ioredis/-/ioredis-5.10.1.tgz",
"integrity": "sha512-HuEDBTI70aYdx1v6U97SbNx9F1+svQKBDo30o0b9fw055LMepzpOOd0Ccg9Q6tbqmBSJaMuY0fB7yw9/vjBYCA==",
"license": "MIT",
"dependencies": {
"@ioredis/commands": "1.5.1",
"cluster-key-slot": "^1.1.0",
"debug": "^4.3.4",
"denque": "^2.1.0",
"lodash.defaults": "^4.2.0",
"lodash.isarguments": "^3.1.0",
"redis-errors": "^1.2.0",
"redis-parser": "^3.0.0",
"standard-as-callback": "^2.1.0"
},
"engines": {
"node": ">=12.22.0"
},
"funding": {
"type": "opencollective",
"url": "https://opencollective.com/ioredis"
}
},
"node_modules/ioredis/node_modules/debug": {
"version": "4.4.3",
"resolved": "https://registry.npmjs.org/debug/-/debug-4.4.3.tgz",
"integrity": "sha512-RGwwWnwQvkVfavKVt22FGLw+xYSdzARwm0ru6DhTVA3umU5hZc28V3kO4stgYryrTlLpuvgI9GiijltAjNbcqA==",
"license": "MIT",
"dependencies": {
"ms": "^2.1.3"
},
"engines": {
"node": ">=6.0"
},
"peerDependenciesMeta": {
"supports-color": {
"optional": true
}
}
},
"node_modules/ioredis/node_modules/ms": {
"version": "2.1.3",
"resolved": "https://registry.npmjs.org/ms/-/ms-2.1.3.tgz",
"integrity": "sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA==",
"license": "MIT"
},
"node_modules/ipaddr.js": {
"version": "1.9.1",
"resolved": "https://registry.npmjs.org/ipaddr.js/-/ipaddr.js-1.9.1.tgz",
@ -1733,6 +1972,27 @@
"node": ">=8"
}
},
"node_modules/lodash.defaults": {
"version": "4.2.0",
"resolved": "https://registry.npmjs.org/lodash.defaults/-/lodash.defaults-4.2.0.tgz",
"integrity": "sha512-qjxPLHd3r5DnsdGacqOMU6pb/avJzdh9tFX2ymgoZE27BmjXrNy/y4LoaiTeAb+O3gL8AfpJGtqfX/ae2leYYQ==",
"license": "MIT"
},
"node_modules/lodash.isarguments": {
"version": "3.1.0",
"resolved": "https://registry.npmjs.org/lodash.isarguments/-/lodash.isarguments-3.1.0.tgz",
"integrity": "sha512-chi4NHZlZqZD18a0imDHnZPrDeBbTtVN7GXMwuGdRH9qotxAjYs3aVLKc7zNOG9eddR5Ksd8rvFEBc9SsggPpg==",
"license": "MIT"
},
"node_modules/luxon": {
"version": "3.7.2",
"resolved": "https://registry.npmjs.org/luxon/-/luxon-3.7.2.tgz",
"integrity": "sha512-vtEhXh/gNjI9Yg1u4jX/0YVPMvxzHuGgCm6tC5kZyb08yjGWGnqAjGJvcXbqQR2P3MyMEFnRbpcdFS6PBcLqew==",
"license": "MIT",
"engines": {
"node": ">=12"
}
},
"node_modules/magic-string": {
"version": "0.30.21",
"resolved": "https://registry.npmjs.org/magic-string/-/magic-string-0.30.21.tgz",
@ -1817,6 +2077,56 @@
"integrity": "sha512-Tpp60P6IUJDTuOq/5Z8cdskzJujfwqfOTkrwIwj7IRISpnkJnT6SyJ4PCPnGMoFjC9ddhal5KVIYtAt97ix05A==",
"license": "MIT"
},
"node_modules/msgpackr": {
"version": "1.11.5",
"resolved": "https://registry.npmjs.org/msgpackr/-/msgpackr-1.11.5.tgz",
"integrity": "sha512-UjkUHN0yqp9RWKy0Lplhh+wlpdt9oQBYgULZOiFhV3VclSF1JnSQWZ5r9gORQlNYaUKQoR8itv7g7z1xDDuACA==",
"license": "MIT",
"optionalDependencies": {
"msgpackr-extract": "^3.0.2"
}
},
"node_modules/msgpackr-extract": {
"version": "3.0.3",
"resolved": "https://registry.npmjs.org/msgpackr-extract/-/msgpackr-extract-3.0.3.tgz",
"integrity": "sha512-P0efT1C9jIdVRefqjzOQ9Xml57zpOXnIuS+csaB4MdZbTdmGDLo8XhzBG1N7aO11gKDDkJvBLULeFTo46wwreA==",
"hasInstallScript": true,
"license": "MIT",
"optional": true,
"dependencies": {
"node-gyp-build-optional-packages": "5.2.2"
},
"bin": {
"download-msgpackr-prebuilds": "bin/download-prebuilds.js"
},
"optionalDependencies": {
"@msgpackr-extract/msgpackr-extract-darwin-arm64": "3.0.3",
"@msgpackr-extract/msgpackr-extract-darwin-x64": "3.0.3",
"@msgpackr-extract/msgpackr-extract-linux-arm": "3.0.3",
"@msgpackr-extract/msgpackr-extract-linux-arm64": "3.0.3",
"@msgpackr-extract/msgpackr-extract-linux-x64": "3.0.3",
"@msgpackr-extract/msgpackr-extract-win32-x64": "3.0.3"
}
},
"node_modules/multer": {
"version": "2.1.1",
"resolved": "https://registry.npmjs.org/multer/-/multer-2.1.1.tgz",
"integrity": "sha512-mo+QTzKlx8R7E5ylSXxWzGoXoZbOsRMpyitcht8By2KHvMbf3tjwosZ/Mu/XYU6UuJ3VZnODIrak5ZrPiPyB6A==",
"license": "MIT",
"dependencies": {
"append-field": "^1.0.0",
"busboy": "^1.6.0",
"concat-stream": "^2.0.0",
"type-is": "^1.6.18"
},
"engines": {
"node": ">= 10.16.0"
},
"funding": {
"type": "opencollective",
"url": "https://opencollective.com/express"
}
},
"node_modules/nanoid": {
"version": "3.3.11",
"resolved": "https://registry.npmjs.org/nanoid/-/nanoid-3.3.11.tgz",
@ -1844,6 +2154,27 @@
"node": ">= 0.6"
}
},
"node_modules/node-abort-controller": {
"version": "3.1.1",
"resolved": "https://registry.npmjs.org/node-abort-controller/-/node-abort-controller-3.1.1.tgz",
"integrity": "sha512-AGK2yQKIjRuqnc6VkX2Xj5d+QW8xZ87pa1UK6yA6ouUyuxfHuMP6umE5QK7UmTeOAymo+Zx1Fxiuw9rVx8taHQ==",
"license": "MIT"
},
"node_modules/node-gyp-build-optional-packages": {
"version": "5.2.2",
"resolved": "https://registry.npmjs.org/node-gyp-build-optional-packages/-/node-gyp-build-optional-packages-5.2.2.tgz",
"integrity": "sha512-s+w+rBWnpTMwSFbaE0UXsRlg7hU4FjekKU4eyAih5T8nJuNZT1nNsskXpxmeqSK9UzkBl6UgRlnKc8hz8IEqOw==",
"license": "MIT",
"optional": true,
"dependencies": {
"detect-libc": "^2.0.1"
},
"bin": {
"node-gyp-build-optional-packages": "bin.js",
"node-gyp-build-optional-packages-optional": "optional.js",
"node-gyp-build-optional-packages-test": "build-test.js"
}
},
"node_modules/object-assign": {
"version": "4.1.1",
"resolved": "https://registry.npmjs.org/object-assign/-/object-assign-4.1.1.tgz",
@ -1991,6 +2322,41 @@
"node": ">= 0.8"
}
},
"node_modules/readable-stream": {
"version": "3.6.2",
"resolved": "https://registry.npmjs.org/readable-stream/-/readable-stream-3.6.2.tgz",
"integrity": "sha512-9u/sniCrY3D5WdsERHzHE4G2YCXqoG5FTHUiCC4SIbr6XcLZBY05ya9EKjYek9O5xOAwjGq+1JdGBAS7Q9ScoA==",
"license": "MIT",
"dependencies": {
"inherits": "^2.0.3",
"string_decoder": "^1.1.1",
"util-deprecate": "^1.0.1"
},
"engines": {
"node": ">= 6"
}
},
"node_modules/redis-errors": {
"version": "1.2.0",
"resolved": "https://registry.npmjs.org/redis-errors/-/redis-errors-1.2.0.tgz",
"integrity": "sha512-1qny3OExCf0UvUV/5wpYKf2YwPcOqXzkwKKSmKHiE6ZMQs5heeE/c8eXK+PNllPvmjgAbfnsbpkGZWy8cBpn9w==",
"license": "MIT",
"engines": {
"node": ">=4"
}
},
"node_modules/redis-parser": {
"version": "3.0.0",
"resolved": "https://registry.npmjs.org/redis-parser/-/redis-parser-3.0.0.tgz",
"integrity": "sha512-DJnGAeenTdpMEH6uAJRK/uiyEIH9WVsUmoLwzudwGJUwZPp80PDBWPHXSAGNPwNvIXAbe7MSUB1zQFugFml66A==",
"license": "MIT",
"dependencies": {
"redis-errors": "^1.0.0"
},
"engines": {
"node": ">=4"
}
},
"node_modules/require-directory": {
"version": "2.1.1",
"resolved": "https://registry.npmjs.org/require-directory/-/require-directory-2.1.1.tgz",
@ -2082,6 +2448,18 @@
"integrity": "sha512-YZo3K82SD7Riyi0E1EQPojLz7kpepnSQI9IyPbHHg1XXXevb5dJI7tpyN2ADxGcQbHG7vcyRHk0cbwqcQriUtg==",
"license": "MIT"
},
"node_modules/semver": {
"version": "7.7.4",
"resolved": "https://registry.npmjs.org/semver/-/semver-7.7.4.tgz",
"integrity": "sha512-vFKC2IEtQnVhpT78h1Yp8wzwrf8CM+MzKMHGJZfBtzhZNycRFnXsHk6E5TxIkkMsgNS7mdX3AGB7x2QM2di4lA==",
"license": "ISC",
"bin": {
"semver": "bin/semver.js"
},
"engines": {
"node": ">=10"
}
},
"node_modules/send": {
"version": "0.19.2",
"resolved": "https://registry.npmjs.org/send/-/send-0.19.2.tgz",
@ -2227,6 +2605,12 @@
"node": ">=0.10.0"
}
},
"node_modules/standard-as-callback": {
"version": "2.1.0",
"resolved": "https://registry.npmjs.org/standard-as-callback/-/standard-as-callback-2.1.0.tgz",
"integrity": "sha512-qoRRSyROncaz1z0mvYqIE4lCd9p2R90i6GxW3uZv5ucSu8tU7B5HXUP1gG8pVZsYNVaXjk8ClXHPttLyxAL48A==",
"license": "MIT"
},
"node_modules/statuses": {
"version": "2.0.2",
"resolved": "https://registry.npmjs.org/statuses/-/statuses-2.0.2.tgz",
@ -2236,6 +2620,23 @@
"node": ">= 0.8"
}
},
"node_modules/streamsearch": {
"version": "1.1.0",
"resolved": "https://registry.npmjs.org/streamsearch/-/streamsearch-1.1.0.tgz",
"integrity": "sha512-Mcc5wHehp9aXz1ax6bZUyY5afg9u2rv5cqQI3mRrYkGC8rW2hM02jWuwjtL++LS5qinSyhj2QfLyNsuc+VsExg==",
"engines": {
"node": ">=10.0.0"
}
},
"node_modules/string_decoder": {
"version": "1.3.0",
"resolved": "https://registry.npmjs.org/string_decoder/-/string_decoder-1.3.0.tgz",
"integrity": "sha512-hkRX8U1WjJFd8LsDJ2yQ/wWWxaopEsABU1XfkM8A+j0+85JAGppt16cr1Whg6KIbb4okU6Mql6BOj+uup/wKeA==",
"license": "MIT",
"dependencies": {
"safe-buffer": "~5.2.0"
}
},
"node_modules/string-width": {
"version": "4.2.3",
"resolved": "https://registry.npmjs.org/string-width/-/string-width-4.2.3.tgz",
@ -2320,7 +2721,6 @@
"version": "2.8.1",
"resolved": "https://registry.npmjs.org/tslib/-/tslib-2.8.1.tgz",
"integrity": "sha512-oJFu94HQb+KVduSUQL7wnpmqnfmLsOA/nAh6b6EH0wCEoK0/mPeXU6c3wKDV83MkOuHPRHtSXKKU99IBazS/2w==",
"dev": true,
"license": "0BSD"
},
"node_modules/type-is": {
@ -2336,6 +2736,12 @@
"node": ">= 0.6"
}
},
"node_modules/typedarray": {
"version": "0.0.6",
"resolved": "https://registry.npmjs.org/typedarray/-/typedarray-0.0.6.tgz",
"integrity": "sha512-/aCDEGatGvZ2BIk+HmLf4ifCJFwvKFNb9/JeZPMulfgFracn9QFcAf5GO8B/mweUjSoblS5In0cWhqpfs/5PQA==",
"license": "MIT"
},
"node_modules/unpipe": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/unpipe/-/unpipe-1.0.0.tgz",
@ -2345,6 +2751,12 @@
"node": ">= 0.8"
}
},
"node_modules/util-deprecate": {
"version": "1.0.2",
"resolved": "https://registry.npmjs.org/util-deprecate/-/util-deprecate-1.0.2.tgz",
"integrity": "sha512-EPD5q1uXyFxJpCrLnCc1nHnq3gOa6DZBocAIiI2TaSCA7VCJ1UJDMagCzIkXNsUYfD1daK//LTEQ8xiIbrHtcw==",
"license": "MIT"
},
"node_modules/utils-merge": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/utils-merge/-/utils-merge-1.0.1.tgz",
@ -2354,6 +2766,19 @@
"node": ">= 0.4.0"
}
},
"node_modules/uuid": {
"version": "11.1.0",
"resolved": "https://registry.npmjs.org/uuid/-/uuid-11.1.0.tgz",
"integrity": "sha512-0/A9rDy9P7cJ+8w1c9WD9V//9Wj15Ce2MPz8Ri6032usz+NfePxx5AcN3bN+r6ZL6jEo066/yNYB3tn4pQEx+A==",
"funding": [
"https://github.com/sponsors/broofa",
"https://github.com/sponsors/ctavan"
],
"license": "MIT",
"bin": {
"uuid": "dist/esm/bin/uuid"
}
},
"node_modules/vary": {
"version": "1.1.2",
"resolved": "https://registry.npmjs.org/vary/-/vary-1.1.2.tgz",

7
faceai/package.json
View file

@ -3,14 +3,17 @@
"private": true,
"workspaces": [
"apps/frontend",
"apps/backend"
"apps/backend",
"apps/processor"
],
"scripts": {
"dev": "concurrently \"npm:dev:backend\" \"npm:dev:frontend\"",
"dev:backend": "npm run dev --workspace @regalami/faceai-backend",
"dev:frontend": "npm run dev --workspace @regalami/faceai-frontend",
"dev:processor": "npm run dev --workspace @regalami/faceai-processor",
"build": "npm run build --workspace @regalami/faceai-frontend && npm run build --workspace @regalami/faceai-backend",
"start": "npm run start --workspace @regalami/faceai-backend"
"start": "npm run start --workspace @regalami/faceai-backend",
"start:processor": "npm run start --workspace @regalami/faceai-processor"
},
"devDependencies": {
"concurrently": "^9.1.2"

BIN
test_pkl/face_encodings_20260330_170155.pkl Normal file
View file

Binary file not shown.

BIN
test_pkl/face_encodings_20260330_170210.pkl Normal file
View file

Binary file not shown.

BIN
test_pkl/face_encodings_20260330_170340.pkl Normal file
View file

Binary file not shown.