maddo/Regalamiunsorriso
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages 1 Wiki Activity Actions

feat(audit): implement audit logging for search requests and results
All checks were successful
Publish FaceAI Container / publish (push) Successful in 13m22s
Details

Browse source 
- Added configuration options for audit database path and retention days in backend and processor.
- Integrated audit logging in server and worker processes to track search requests, completions, and failures.
- Created utility functions for reading and parsing audit logs in end-to-end tests.
- Updated Docker Compose files to include audit database configuration.
- Added new tests to verify audit log entries for successful and no-results searches.
This commit is contained in:
MaddoScientisto 2026-05-19 23:29:38 +02:00
commit 32db61c381
14 changed files with 1067 additions and 16 deletions
Download patch file Download diff file Expand all files Collapse all files

17
faceai/README.md
View file

@ -103,11 +103,22 @@ After `docker compose --env-file .env.development up --build`, inspect:
- `faceai/logs/backend.log` for backend startup and API-side failures
- `faceai/logs/processor.log` for worker startup, queue processing, and uncaught processor errors
- `faceai/logs/faceai-audit.sqlite` for the structured 24-month audit trail of FaceAI usage
- `faceai/logs/searches/<searchId>/worker.log` for the per-search processor trace
- `faceai/logs/searches/<searchId>/matcher.log` for the native `face_matcher` output
This keeps the useful processor diagnostics outside the Docker-managed runtime volume so they survive container rebuilds and can be inspected directly from the workspace.
The audit database is a lightweight SQLite file shared by the public FaceAI service and the processor on the existing log volume. Each search row stores the requesting user, race metadata, request timestamp, request IP and user agent, the uploaded selfie SHA-256 fingerprint, and the final match snapshot. That makes the log queryable without introducing another service and lets you recover the same result set a user originally saw by looking up `selfie_sha256`.
The default retention is 730 days, matching the requested 24-month window. Old audit rows are pruned automatically by FaceAI during normal runtime.
Example query:
```bash
sqlite3 faceai/logs/faceai-audit.sqlite "SELECT search_id, user_id, race_id, requested_at, match_count FROM faceai_audit_searches WHERE selfie_sha256 = '...';"
```
Because the service entrypoints now mirror output instead of redirecting it away, the same startup and runtime messages are also visible through `docker logs regalami-faceai`, `docker logs regalami-faceai-processor`, and Portainer's container log viewer.
The current bundled Linux `face_matcher` binary is a PyInstaller build that requires `GLIBC_2.38` or newer and the `libxcb.so.1` runtime library. The checked-in local processor image satisfies that requirement.
@ -307,6 +318,8 @@ Shared application settings:
| `FACEAI_QUEUE_NAME` | optional | `faceai-searches` | BullMQ queue name |
| `FACEAI_RUNTIME_ROOT` | yes | `/data/runtime` | shared writable runtime root between site and processor |
| `FACEAI_LOG_ROOT` | recommended | `/data/logs` | persistent host-mounted diagnostics root for backend, processor, and per-search logs |
| `FACEAI_AUDIT_DB_PATH` | recommended | `/data/logs/faceai-audit.sqlite` | SQLite audit database shared by backend and processor |
| `FACEAI_AUDIT_RETENTION_DAYS` | recommended | `730` | how long structured audit rows are kept before automatic pruning |
| `FACEAI_SHARED_SECRET` | yes | long random secret | trust boundary between FaceAI and the legacy bridge |
Public site settings:
@ -328,7 +341,7 @@ Processor settings:
| --- | --- | --- | --- |
| `FACEAI_PKL_ROOT` | yes | `/data/pkl` | mounted race-to-PKL dataset root |
| `FACEAI_MATCHER_BINARY` | yes | `/app/bin/face_matcher` | matcher executable baked into the processor image |
| `FACEAI_MATCHER_TOLERANCE` | optional | `0.5` | forwarded to `face_matcher --tollerance`; must stay between `0.35` and `0.75` |
| `FACEAI_MATCHER_TOLERANCE` | optional | `0.5` | forwarded to `face_matcher --tolerance`; must stay between `0.35` and `0.75` |
| `FACEAI_WORKER_CONCURRENCY` | optional | `2` | BullMQ worker concurrency |
| `FACEAI_WORKER_TIMEOUT_MS` | optional | `300000` | matcher timeout in milliseconds |
@ -431,6 +444,8 @@ FACEAI_QUEUE_NAME=faceai-searches
FACEAI_RUNTIME_ROOT=/data/runtime
FACEAI_UPLOAD_ROOT=/data/runtime/uploads
FACEAI_LOG_ROOT=/data/logs
FACEAI_AUDIT_DB_PATH=/data/logs/faceai-audit.sqlite
FACEAI_AUDIT_RETENTION_DAYS=730
FACEAI_PKL_ROOT=/data/pkl
FACEAI_MATCHER_BINARY=/app/bin/face_matcher
FACEAI_MATCHER_TOLERANCE=0.5