Refactor FaceAI build process and update Docker configurations to include matcher binary
Some checks failed
Publish FaceAI Container / publish (push) Failing after 1m17s

This commit is contained in:
MaddoScientisto 2026-04-19 11:00:50 +02:00
commit 774d304220
15 changed files with 177 additions and 30 deletions

6
.dockerignore Normal file
View file

@ -0,0 +1,6 @@
*
!faceai/
!faceai/**
!bin/
!bin/Face_Recognition_Unix/
!bin/Face_Recognition_Unix/**

View file

@ -6,6 +6,7 @@ on:
- master
paths:
- faceai/**
- bin/Face_Recognition_Unix/**
- .forgejo/workflows/publish-faceai-container.yml
workflow_dispatch:
@ -13,7 +14,7 @@ env:
REGISTRY: ${{ vars.FORGEJO_REGISTRY }}
IMAGE_NAMESPACE: ${{ vars.IMAGE_NAMESPACE }}
IMAGE_NAME: ${{ vars.IMAGE_NAME != '' && vars.IMAGE_NAME || 'faceai' }}
BUILD_CONTEXT: faceai
BUILD_CONTEXT: .
DOCKERFILE_PATH: faceai/docker/Dockerfile
jobs:
@ -33,7 +34,8 @@ jobs:
if [ -z "${IMAGE_NAMESPACE}" ]; then echo "vars.IMAGE_NAMESPACE is required"; exit 1; fi
if [ -z "${IMAGE_NAME}" ]; then echo "vars.IMAGE_NAME resolved to an empty value"; exit 1; fi
if [ ! -f "${DOCKERFILE_PATH}" ]; then echo "Dockerfile not found at ${DOCKERFILE_PATH}"; exit 1; fi
if [ ! -f "${BUILD_CONTEXT}/package.json" ]; then echo "Build context is invalid: ${BUILD_CONTEXT}"; exit 1; fi
if [ ! -f "faceai/package.json" ]; then echo "faceai/package.json is missing from the repository checkout"; exit 1; fi
if [ ! -f "bin/Face_Recognition_Unix/face_matcher" ]; then echo "bin/Face_Recognition_Unix/face_matcher is missing from the repository checkout"; exit 1; fi
- name: Validate registry secrets
run: |

View file

@ -75,12 +75,11 @@ The checked-in `docker-compose.yml` starts:
The local stack also mounts:
- `../bin/Face_Recognition_Unix` into the processor container as the matcher binary source
- `../test_pkl` into both the public FaceAI container and the processor container as the shared read-only PKL dataset root
- `./logs` into both the public FaceAI container and the processor container as the persistent diagnostics directory
- `../www` into the PHP container so the real bridge files are used
The `processor` service is built from `docker/processor.Dockerfile`, which uses a Debian Trixie-based Node 22 image, applies the current package upgrades available during build, and installs `libxcb1` so the bundled Linux `face_matcher` binary can run locally.
The `processor` service is built from `docker/processor.Dockerfile` using the repository root as Docker build context. That image copies the checked-in `bin/Face_Recognition_Unix` directory into `/opt/face-recognition` during `docker build`, so the matcher binary is baked into the image instead of being mounted from the host at runtime.
### Persistent Logs
@ -212,9 +211,23 @@ LIVE_SITE_RUN_UPLOAD_FLOW=1
When enabled, the live suite also:
- validates that the legacy Face ID handoff URL includes the race storage metadata expected by FaceAI
- opens the real FaceAI app and asserts that the legacy header stylesheets load from the live legacy site
- opens the real FaceAI app and asserts that the legacy header stylesheets load from the live legacy site without injecting cross-origin Font Awesome assets
- confirms the app does not emit the `MISSING_RACE_STORAGE` invalid-race error on launch
- uploads the supplied portrait image and verifies that search creation succeeds
- uploads the supplied portrait image, waits for the search to complete, and requires a redirect back to the legacy result page with rendered results
### Processor Troubleshooting
If the processor logs show an error like `spawn /opt/face-recognition/face_matcher ENOENT`, the problem is not the upload flow itself. It means the running processor cannot see the matcher binary at the configured `FACEAI_MATCHER_BINARY` path.
With the current checked-in Dockerfiles, the matcher binary is copied into the image from the repository source tree during `docker build`. The runtime container no longer needs a host bind mount for `/opt/face-recognition`.
Published images now get that binary because the Forgejo container workflow builds from the repository root, which lets `faceai/docker/Dockerfile` copy:
```text
bin/Face_Recognition_Unix/face_matcher
```
If a running processor still reports `ENOENT`, the deployed image was built before this change or the build did not include the checked-in matcher directory.
## Optional Backend And Frontend Dev Loop
@ -320,7 +333,6 @@ services:
- /mnt/storage/data/faceai/runtime:/data/runtime
- /mnt/storage/data/faceai/logs:/data/logs
- /mnt/nas12/nas2/RUS:/data/pkl:ro
- /mnt/storage/data/faceai/bin/Face_Recognition_Unix:/opt/face-recognition:ro
depends_on:
redis:
condition: service_healthy
@ -372,7 +384,7 @@ Processor settings:
| Variable | Required | Example | Purpose |
| --- | --- | --- | --- |
| `FACEAI_PKL_ROOT` | yes | `/data/pkl` | mounted race-to-PKL dataset root |
| `FACEAI_MATCHER_BINARY` | yes | `/opt/face-recognition/face_matcher` | matcher executable inside the processor container |
| `FACEAI_MATCHER_BINARY` | yes | `/opt/face-recognition/face_matcher` | matcher executable baked into the image |
| `FACEAI_WORKER_CONCURRENCY` | optional | `2` | BullMQ worker concurrency |
| `FACEAI_WORKER_TIMEOUT_MS` | optional | `300000` | matcher timeout in milliseconds |

View file

@ -117,7 +117,10 @@ const emit = defineEmits([
<div class="faceai-dropzone-inner">
<div class="faceai-dropzone-icon">
<i class="fa fa-cloud-upload" aria-hidden="true"></i>
<svg viewBox="0 0 24 24" aria-hidden="true" focusable="false">
<path d="M12.75 3.75a.75.75 0 0 0-1.5 0v8.69L8.78 9.97a.75.75 0 1 0-1.06 1.06l3.75 3.75a.75.75 0 0 0 1.06 0l3.75-3.75a.75.75 0 1 0-1.06-1.06l-2.47 2.47V3.75Z" fill="currentColor" />
<path d="M5.25 15a3 3 0 0 0 .28 6h12.94a3.53 3.53 0 0 0 .27-7.04 5.75 5.75 0 0 0-11.17-1.48A3.75 3.75 0 0 0 5.25 15Zm1.5 0c0-.87.35-1.65.92-2.22a.75.75 0 0 0 .18-.77 4.25 4.25 0 0 1 8.29 1.39v.25a.75.75 0 0 0 .75.75h1.58a2.03 2.03 0 0 1 0 4.06H5.53A1.5 1.5 0 1 1 5.53 15h1.22Z" fill="currentColor" />
</svg>
</div>
<template v-if="selectedFile">
@ -290,6 +293,11 @@ const emit = defineEmits([
box-shadow: inset 0 0 0 1px rgba(191, 158, 117, 0.24);
}
.faceai-dropzone-icon svg {
width: 44px;
height: 44px;
}
.faceai-dropzone-title {
font-size: 1.2rem;
font-weight: 700;

View file

@ -64,7 +64,12 @@ function closeMenu() {
<ul class="navbar-nav ml-auto">
<li class="nav-item">
<a class="nav-link active" href="#" @click="closeMenu">
<i class="fa fa-user" aria-hidden="true"></i> Il mio account
<span class="faceai-inline-icon" aria-hidden="true">
<svg viewBox="0 0 24 24" focusable="false">
<path d="M12 12.5a4.25 4.25 0 1 0 0-8.5 4.25 4.25 0 0 0 0 8.5Zm0 2.25c-4.48 0-8.25 2.18-8.25 4.75 0 .28.22.5.5.5h15.5a.5.5 0 0 0 .5-.5c0-2.57-3.77-4.75-8.25-4.75Z" fill="currentColor" />
</svg>
</span>
Il mio account
</a>
</li>
<li class="nav-item">
@ -78,3 +83,18 @@ function closeMenu() {
</nav>
</div>
</template>
<style scoped>
.faceai-inline-icon {
display: inline-flex;
width: 1rem;
height: 1rem;
margin-right: 0.35rem;
vertical-align: text-bottom;
}
.faceai-inline-icon svg {
width: 100%;
height: 100%;
}
</style>

View file

@ -36,7 +36,6 @@ export function legacyAsset(path) {
export function injectLegacyStylesheets() {
const stylesheets = [
legacyAsset('/vendor/bootstrap/css/bootstrap.min.css'),
legacyAsset('/css/font-awesome.min.css'),
'https://fonts.googleapis.com/css?family=Roboto:300,300i,400,400i,700,700i',
legacyAsset('/css/custom-style.css')
];

View file

@ -40,6 +40,10 @@ export async function runFaceMatcher({ matcherBinary, selfiePath, pklPath, csvPa
child.on('error', (error) => {
clearTimeout(timer);
if (error?.code === 'ENOENT') {
reject(new Error(`face_matcher not found at ${matcherBinary}. Check FACEAI_MATCHER_BINARY and the processor bind mount.`));
return;
}
reject(error);
});

View file

@ -1,3 +1,4 @@
import { constants as fsConstants } from 'node:fs';
import fs from 'node:fs/promises';
import path from 'node:path';
import { Worker } from 'bullmq';
@ -15,6 +16,22 @@ import { parseMatcherCsv, resolvePklPath, runFaceMatcher } from './worker-utils.
const connection = createRedisConnection(config.redisUrl);
async function ensureMatcherBinaryAvailable() {
try {
await fs.access(config.matcherBinary, fsConstants.X_OK);
} catch (error) {
console.error(`FaceAI processor cannot start because the matcher binary is unavailable: ${config.matcherBinary}`);
console.error('Ensure FACEAI_MATCHER_BINARY points to a real executable and that the processor bind mount contains face_matcher.');
if (error?.code === 'ENOENT') {
console.error('The configured matcher path does not exist inside the processor runtime.');
} else if (error?.code === 'EACCES') {
console.error('The configured matcher path exists but is not executable by the processor runtime.');
}
throw error;
}
}
function formatLogLine(message, details) {
const timestamp = new Date().toISOString();
if (details === undefined) {
@ -147,6 +164,8 @@ async function processJob(job) {
}
}
await ensureMatcherBinaryAvailable();
const worker = new Worker(config.queueName, processJob, {
connection,
concurrency: config.workerConcurrency

View file

@ -48,8 +48,8 @@ services:
processor:
build:
context: .
dockerfile: docker/processor.Dockerfile
context: ..
dockerfile: faceai/docker/processor.Dockerfile
image: regalami-faceai-processor-local
container_name: regalami-faceai-processor
restart: unless-stopped
@ -74,7 +74,6 @@ services:
volumes:
- .:/app
- ./logs:/data/logs
- ../bin/Face_Recognition_Unix:/opt/face-recognition:ro
- ../test_pkl:/data/pkl:ro
- faceai-runtime:/data/runtime
depends_on:

View file

@ -1,25 +1,40 @@
FROM node:20-alpine AS build
FROM node:22-trixie-slim AS build
RUN apt-get update \
&& apt-get upgrade -y \
&& apt-get install -y --no-install-recommends libxcb1 \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /app
COPY package.json ./
COPY apps/frontend/package.json apps/frontend/package.json
COPY apps/backend/package.json apps/backend/package.json
COPY apps/processor/package.json apps/processor/package.json
COPY faceai/package.json ./package.json
COPY faceai/apps/frontend/package.json apps/frontend/package.json
COPY faceai/apps/backend/package.json apps/backend/package.json
COPY faceai/apps/processor/package.json apps/processor/package.json
RUN npm install
COPY . .
COPY faceai/ .
COPY bin/Face_Recognition_Unix /opt/face-recognition
RUN chmod +x /opt/face-recognition/face_matcher
RUN npm run build
FROM node:20-alpine AS runtime
FROM node:22-trixie-slim AS runtime
RUN apt-get update \
&& apt-get upgrade -y \
&& apt-get install -y --no-install-recommends libxcb1 \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /app
COPY --from=build /app /app
COPY --from=build /opt/face-recognition /opt/face-recognition
ENV NODE_ENV=production
ENV FACEAI_MATCHER_BINARY=/opt/face-recognition/face_matcher
EXPOSE 3001
CMD ["npm", "run", "start"]

View file

@ -5,4 +5,11 @@ RUN apt-get update \
&& apt-get install -y --no-install-recommends libxcb1 \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /app
WORKDIR /app
COPY faceai /app
COPY bin/Face_Recognition_Unix /opt/face-recognition
RUN chmod +x /opt/face-recognition/face_matcher
ENV FACEAI_MATCHER_BINARY=/opt/face-recognition/face_matcher

View file

@ -4,6 +4,7 @@ const {
LIVE_SITE_BASE_URL,
LIVE_SITE_PORTRAIT_PATH,
LIVE_SITE_RACE_URL,
LIVE_SITE_RESULT_URL_PATTERN,
LIVE_SITE_RUN_UPLOAD_FLOW,
ensureLiveAuthenticatedRacePage,
requirePortraitFixture
@ -51,6 +52,34 @@ async function openLiveFaceAi(page) {
};
}
async function waitForSearchCompletion(page, searchId) {
return expect.poll(async () => {
return page.evaluate(async (id) => {
const response = await fetch(`/api/searches/${id}`, { credentials: 'include' });
if (!response.ok) {
return {
status: 'poll-error',
httpStatus: response.status
};
}
const payload = await response.json();
return {
status: payload.status,
errorMessage: payload.errorMessage || null,
completionCode: payload.completionCode || null,
matchCount: payload.matchCount ?? null,
resultId: payload.resultId || null
};
}, searchId);
}, {
timeout: 3 * 60 * 1000,
message: `Expected FaceAI search ${searchId} to complete successfully.`
}).toMatchObject({
status: 'completed'
});
}
test('loads a live race page with an authenticated session', async ({ page }) => {
await ensureLiveAuthenticatedRacePage(page);
@ -70,6 +99,7 @@ test('launches the live FaceAI app with race storage metadata and a styled heade
await expect(page.locator('nav.navbar')).toBeVisible();
await expect(page.locator('link[data-legacy-href*="bootstrap.min.css"]')).toHaveCount(1);
await expect(page.locator('link[data-legacy-href*="custom-style.css"]')).toHaveCount(1);
await expect(page.locator('link[data-legacy-href*="font-awesome"]')).toHaveCount(0);
const legacyStylesheetHrefs = await page.locator('link[data-legacy-href]').evaluateAll((elements) => {
return elements.map((element) => element.getAttribute('href') || '');
@ -116,6 +146,32 @@ test('accepts the supplied portrait image for the live upload flow', async ({ pa
expect(searchResponse.ok(), 'Expected the live upload flow to create a FaceAI search successfully.').toBe(true);
const searchPayload = await searchResponse.json();
expect(searchPayload.id || searchPayload.searchId, 'Expected the search creation response to include a search identifier.').toBeTruthy();
await expect(page.locator('.faceai-feedback')).not.toContainText(/Impossibile|Unable|Errore|Error/i);
const searchId = searchPayload.id || searchPayload.searchId;
expect(searchId, 'Expected the search creation response to include a search identifier.').toBeTruthy();
const completion = await page.evaluate(async (id) => {
const response = await fetch(`/api/searches/${id}`, { credentials: 'include' });
return response.json();
}, searchId).catch(() => null);
if (completion?.status === 'failed') {
throw new Error(`FaceAI search ${searchId} failed immediately: ${completion.errorMessage || 'unknown error'}`);
}
await waitForSearchCompletion(page, searchId);
await page.waitForURL(new RegExp(`^${escapeRegExp(LIVE_SITE_RESULT_URL_PATTERN)}`), {
timeout: 3 * 60 * 1000
});
await expect.poll(async () => page.url(), {
timeout: 15 * 1000,
message: 'Expected the browser to land on the legacy result page after FaceAI completed.'
}).toMatch(new RegExp(`^${escapeRegExp(LIVE_SITE_BASE_URL)}/`));
await expect(page.locator('body')).toContainText(/Vista filtrata da FaceAI|foto da FaceAI|ID foto:/i);
await expect.poll(async () => page.locator('.gallery-card').count(), {
timeout: 15 * 1000,
message: 'Expected the legacy return page to render at least one FaceAI result card.'
}).toBeGreaterThan(0);
});

View file

@ -7,6 +7,7 @@ const LIVE_SITE_BASE_URL = process.env.LIVE_SITE_BASE_URL || 'https://www.regala
const LIVE_SITE_LOGIN_URL = process.env.LIVE_SITE_LOGIN_URL || `${LIVE_SITE_BASE_URL}/login_clienti-it.html`;
const LIVE_SITE_RACE_URL = process.env.LIVE_SITE_RACE_URL || `${LIVE_SITE_BASE_URL}/42%20HALF%20MARATHON%20FIRENZE_gara-1018545---96-1.html`;
const LIVE_FACEAI_BASE_URL = process.env.LIVE_FACEAI_BASE_URL || 'https://ai.regalamiunsorriso.it';
const LIVE_SITE_RESULT_URL_PATTERN = process.env.LIVE_SITE_RESULT_URL_PATTERN || `${LIVE_SITE_BASE_URL}/faceai_return.php`;
const LIVE_SITE_USERNAME = process.env.LIVE_SITE_USERNAME || '';
const LIVE_SITE_PASSWORD = process.env.LIVE_SITE_PASSWORD || '';
const LIVE_SITE_PORTRAIT_PATH = process.env.LIVE_SITE_PORTRAIT_PATH || path.join(WORKSPACE_ROOT, 'test_pkl', 'live', 'test_portrait_1.png');
@ -109,6 +110,7 @@ module.exports = {
LIVE_SITE_PASSWORD,
LIVE_SITE_PORTRAIT_PATH,
LIVE_SITE_RACE_URL,
LIVE_SITE_RESULT_URL_PATTERN,
LIVE_SITE_RUN_UPLOAD_FLOW,
LIVE_SITE_USERNAME,
dismissCookieBanner,

View file

@ -48,7 +48,6 @@ services:
- /var/docker/faceai/runtime:/data/runtime
- /var/docker/faceai/logs:/data/logs
- /mnt/nas12/nas2/RUS:/data/pkl:ro
- /var/docker/faceai/bin/Face_Recognition_Unix:/opt/face-recognition:ro
depends_on:
- redis

View file

@ -10,13 +10,12 @@ All files in this rollout are deployed from the current working tree.
## New Files
- `www/_inc_faceai_identity.jsp`
- `www/_js/lang.js`
- None in this rollout.
## Updated Files
- `www/fotoCR-en.jsp`
- `www/fotoCR.jsp`
- `www/_js/rus-ecom-240621.js`
- `www/faceai_handoff.php`
## Excluded Files
@ -28,7 +27,7 @@ All files in this rollout are deployed from the current working tree.
- Remote host: `marco@83.149.164.4:410`
- Remote staging path: `/home/marco/regalamiunsorriso/incoming/www`
- Remote live path: `/home/sites/regalamiunsorriso/www`
- Total files in this manifest: `4`
- Total files in this manifest: `2`
## Transfer Method