Enhance Docker and PowerShell scripts for improved functionality and maintainability
All checks were successful
Publish FaceAI Container / publish (push) Successful in 6m52s
All checks were successful
Publish FaceAI Container / publish (push) Successful in 6m52s
- Updated Dockerfile to include default MySQL client for better database interaction. - Modified entrypoint.sh to support additional workspace for legacy applications and added MySQL readiness check before startup. - Enhanced PowerShell script for trimming MySQL dumps to include overlay dumps and improved error handling for missing race and user IDs. - Added new image files and face encoding pickles for various projects, ensuring comprehensive data availability. - Removed outdated face encoding pickle from PISA directory to maintain data relevance. Co-authored-by: Copilot <copilot@github.com>
This commit is contained in:
parent
c227fce036
commit
dd7d4c865b
54 changed files with 492 additions and 144 deletions
|
|
@ -7,8 +7,7 @@ It includes:
|
|||
- a Vue frontend for the FaceAI upload and polling flow
|
||||
- a Node/Express backend for session exchange, queueing, and return handoff
|
||||
- a dedicated processor runner that consumes matcher jobs from Redis and executes `face_matcher`
|
||||
- a local legacy simulator so the launch and return flow can be tested without the old Java site
|
||||
- a Dockerized PHP Apache stack for exercising the real `www/faceai_handoff.php` and `www/faceai_return.php` bridge files
|
||||
- a local Dockerized Tomcat/JSP stack so the launch and return flow can be tested against the real legacy race pages under `www/`
|
||||
|
||||
## Structure
|
||||
|
||||
|
|
@ -24,32 +23,38 @@ faceai/
|
|||
|
||||
## Runtime Topology
|
||||
|
||||
The scaffold currently expects four runtime roles:
|
||||
The production-oriented application topology is still three services:
|
||||
|
||||
- `faceai`: public HTTP service on port `3001`, serving the built Vue app and the authenticated API
|
||||
- `processor`: background matcher runner consuming BullMQ jobs from Redis and executing the Linux `face_matcher` binary
|
||||
- `redis`: short-lived queue and search-state store
|
||||
- `legacy-php`: local-only PHP Apache simulator for exercising the real bridge files under `www/`
|
||||
|
||||
For hosted deployment, the long-lived application topology is `faceai` + `processor` + `redis`. The PHP simulator stays local-only and the real legacy site remains on its existing stack.
|
||||
The checked-in development override now expands that into the full local integration stack:
|
||||
|
||||
- `tomcat-www`: local Tomcat runtime serving the real legacy JSP race pages from `www/`
|
||||
- `mysql`: local legacy database used by the Tomcat stack
|
||||
- `maildump`: local SMTP sink and viewer for the Tomcat stack
|
||||
|
||||
That means the local FaceAI flow now runs against the same legacy page type the real site uses instead of a separate PHP bridge container.
|
||||
|
||||
## What The End-To-End Local Test Covers
|
||||
|
||||
The local simulator exercises the exact flow the plan is aiming for:
|
||||
The local Tomcat stack exercises the exact flow the current integration is aiming for:
|
||||
|
||||
1. a legacy-like race page loads the original `www/_js/rus-ecom-240621.js` script and shows a `Face ID` button instead of `tipoPuntoFoto`
|
||||
2. clicking it hits the real PHP handoff bridge at `www/faceai_handoff.php`
|
||||
2. clicking it uses the handoff URL emitted by the real JSP race page and launches FaceAI through `http://localhost:3001/dev/legacy/launch`
|
||||
3. the backend signs a short-lived handoff token and redirects to the Vue app
|
||||
4. the Vue app exchanges the token for its own FaceAI session cookie
|
||||
5. the user uploads a selfie and starts a Redis-backed race-scoped search
|
||||
6. the frontend polls until the job completes
|
||||
7. FaceAI requests a signed return URL
|
||||
8. the browser is redirected back to the real PHP return bridge at `www/faceai_return.php`
|
||||
9. the PHP bridge fetches the signed result from FaceAI and renders a filtered legacy-like race page
|
||||
8. the browser is redirected back to the real Tomcat race page with `faceaiResultId` and `faceaiToken`
|
||||
9. the legacy race-page JavaScript fetches the signed result from FaceAI, stores the hydrated match payload in browser storage, and reloads the cleaned race URL
|
||||
10. the same legacy race page renders the FaceAI-filtered gallery view from that stored payload
|
||||
|
||||
## Local Testing With The Legacy PHP Simulator
|
||||
## Local Testing With The Merged Legacy JSP Stack
|
||||
|
||||
This is the recommended local test path because it exercises the public site, the processor, Redis, and the real PHP bridge files together.
|
||||
This is the recommended local test path because it exercises the public site, the processor, Redis, and the real legacy JSP page flow together.
|
||||
|
||||
### Prerequisites
|
||||
|
||||
|
|
@ -78,13 +83,15 @@ The local development stack started by the command above combines the base file
|
|||
- FaceAI public site on `http://localhost:3001`
|
||||
- processor runner on the internal Compose network
|
||||
- Redis on the internal Compose network
|
||||
- PHP Apache serving `../www` on `http://localhost:8080`
|
||||
- Tomcat serving the real local legacy site on `http://localhost:8080`
|
||||
- MySQL for the local legacy stack
|
||||
- Maildump for the local legacy stack on `http://localhost:8025`
|
||||
|
||||
The local stack also mounts:
|
||||
|
||||
- `../test_pkl` into both the public FaceAI container and the processor container as the shared read-only PKL dataset root
|
||||
- `./logs` into both the public FaceAI container and the processor container as the persistent diagnostics directory
|
||||
- `../www` into the PHP container so the real bridge files are used
|
||||
- `../www` into the Tomcat build/runtime inputs so the real legacy JSP pages and assets are used
|
||||
|
||||
The `processor` service is built from `docker/processor.Dockerfile` using the repository root as Docker build context. That image copies only the checked-in Unix `face_matcher` into the image, so the matcher is baked into the processor runtime without bringing along the other Unix or Windows binaries.
|
||||
|
||||
|
|
@ -110,22 +117,23 @@ The current bundled Linux `face_matcher` binary is a PyInstaller build that requ
|
|||
Open:
|
||||
|
||||
```text
|
||||
http://localhost:8080/faceai_simulator.php?raceId=202&lang=it
|
||||
http://localhost:8080/Foto2.abl?id_gara=1018557&pageRow=96&pageNumber=1
|
||||
```
|
||||
|
||||
That page simulates the legacy race page, loads the original race-page JavaScript from `www/_js/rus-ecom-240621.js`, lets the script replace the visible `tipoPuntoFoto` selector with the new `Face ID` button, and launches the real PHP handoff bridge at `www/faceai_handoff.php`.
|
||||
That is the real local Tomcat race page. It loads the original race-page JavaScript from `www/_js/rus-ecom-240621.js`, lets the script replace the visible `tipoPuntoFoto` selector with the new `Face ID` button, and launches the backend handoff endpoint configured through the JSP page.
|
||||
|
||||
### Expected Local Flow
|
||||
|
||||
Use the page above and verify this sequence:
|
||||
|
||||
1. the simulator page renders on port `8080`
|
||||
1. the real local race page renders on port `8080`
|
||||
2. the visible checkpoint selector is replaced with the `Face ID` launch button
|
||||
3. clicking `Face ID` redirects through `faceai_handoff.php` into `http://localhost:3001/auth/callback?token=...`
|
||||
3. clicking `Face ID` redirects through `http://localhost:3001/dev/legacy/launch` into `http://localhost:3001/auth/callback?token=...`
|
||||
4. the FaceAI app establishes its session and loads the upload flow
|
||||
5. uploading a selfie creates a queued search that the processor picks up
|
||||
6. when polling completes, FaceAI redirects back to `http://localhost:8080/faceai_return.php?...`
|
||||
7. the PHP return page renders the filtered photo list from the FaceAI result payload
|
||||
6. when polling completes, FaceAI redirects back to the same race page with `faceaiResultId` and `faceaiToken`
|
||||
7. the race-page JavaScript hydrates the result from FaceAI and reloads the cleaned legacy URL
|
||||
8. the filtered photo list renders from the hydrated FaceAI result payload
|
||||
|
||||
### Rebuild Notes
|
||||
|
||||
|
|
@ -143,7 +151,7 @@ docker compose --env-file .env.development down
|
|||
|
||||
### Automated End-To-End Test
|
||||
|
||||
The workspace now includes a Playwright suite that drives the PHP simulator, the FaceAI app, and the processor end to end.
|
||||
The workspace now includes a Playwright suite that drives the real local Tomcat race page, the FaceAI app, and the processor end to end.
|
||||
|
||||
From this folder, run:
|
||||
|
||||
|
|
@ -157,10 +165,10 @@ The suite will:
|
|||
|
||||
- build the frontend bundle
|
||||
- start `docker compose --env-file .env.development up --build -d`
|
||||
- open `http://localhost:8080/faceai_simulator.php?raceId=202&lang=it`
|
||||
- open `http://localhost:8080/Foto2.abl?id_gara=1018557&pageRow=96&pageNumber=1`
|
||||
- click the `Face ID` launch button injected by `www/_js/rus-ecom-240621.js`
|
||||
- upload `test_pkl/test_images/DSC_1960.JPG`
|
||||
- wait for the processor to complete and for FaceAI to redirect to `faceai_return.php`
|
||||
- wait for the processor to complete and for FaceAI to redirect back to the legacy race page
|
||||
- assert the filtered legacy result contains the expected `6` matches and includes `DSC_1960.JPG`
|
||||
- validate `faceai/logs/backend.log`, `faceai/logs/processor.log`, and the per-search `worker.log` and `matcher.log` for the run
|
||||
- stop the Compose stack automatically when the suite finishes
|
||||
|
|
@ -238,7 +246,7 @@ If a running processor still reports `ENOENT`, the deployed image was built befo
|
|||
|
||||
## Optional Backend And Frontend Dev Loop
|
||||
|
||||
If you only want to iterate on the app without the PHP simulator, you can still run the public site and the processor separately. The queue-backed flow now requires Redis and the processor, so `npm run dev` alone is no longer the full stack.
|
||||
If you only want to iterate on the app without the local Tomcat stack, you can still run the public site and the processor separately. The queue-backed flow now requires Redis and the processor, so `npm run dev` alone is no longer the full stack.
|
||||
|
||||
One workable loop is:
|
||||
|
||||
|
|
@ -252,7 +260,7 @@ Then start the processor in a second shell, either with its own local environmen
|
|||
|
||||
## Docker Compose Deployment For The Public Site And Matcher Runner
|
||||
|
||||
The checked-in `docker-compose.yml` is now the production-ready base stack for hosted deployment. The checked-in `docker-compose.override.yml` is the development overlay that restores the local PHP simulator, workspace bind mounts, and development-oriented commands.
|
||||
The checked-in `docker-compose.yml` is now the production-ready base stack for hosted deployment. The checked-in `docker-compose.override.yml` is the development overlay that restores the local Tomcat/JSP stack, workspace bind mounts, and development-oriented commands.
|
||||
|
||||
Because Docker Compose auto-loads `docker-compose.override.yml` when it is present in the same directory, production-style runs from this workspace must explicitly select only the base file.
|
||||
|
||||
|
|
@ -367,8 +375,8 @@ After the Compose stack is up, validate at least the following:
|
|||
2. The legacy handoff endpoint redirects to `https://faceai.../auth/callback?token=...`.
|
||||
3. FaceAI can exchange the token and establish a session.
|
||||
4. A search is enqueued in Redis and picked up by the processor.
|
||||
5. Completing a search produces a redirect URL that points to `FACEAI_LEGACY_RETURN_URL`.
|
||||
6. The legacy return endpoint can resolve the signed result and render the filtered race page.
|
||||
5. Completing a search produces a redirect URL that points to the intended legacy return target.
|
||||
6. The legacy page can resolve the signed result and render the filtered race page.
|
||||
|
||||
### Current Production Limitations
|
||||
|
||||
|
|
@ -411,9 +419,10 @@ NODE_ENV=development
|
|||
FACEAI_PORT=3001
|
||||
FACEAI_FRONTEND_URL=http://localhost:3001
|
||||
FACEAI_PUBLIC_BASE_URL=http://localhost:3001
|
||||
FACEAI_LEGACY_RETURN_URL=http://localhost:8080/faceai_return.php
|
||||
FACEAI_LEGACY_RETURN_MODE=direct
|
||||
FACEAI_LEGACY_HOME_URL=http://localhost:8080/index.jsp
|
||||
FACEAI_FEATURE_ENABLED=1
|
||||
FACEAI_HANDOFF_URL=http://localhost:3001/dev/legacy/launch
|
||||
FACEAI_SHARED_SECRET=change-me
|
||||
FACEAI_SESSION_COOKIE=rus_faceai_session
|
||||
FACEAI_REDIS_URL=redis://redis:6379
|
||||
|
|
@ -425,12 +434,13 @@ FACEAI_PKL_ROOT=/data/pkl
|
|||
FACEAI_MATCHER_BINARY=/app/bin/face_matcher
|
||||
```
|
||||
|
||||
If you want FaceAI to return through the new PHP bridge prepared under `www`, point `FACEAI_LEGACY_RETURN_URL` to that endpoint instead, for example `http://localhost/faceai_return.php` or the equivalent URL in your local PHP setup.
|
||||
If you want FaceAI to force the older bridge-style return path instead, set `FACEAI_LEGACY_RETURN_MODE=bridge` and point `FACEAI_LEGACY_RETURN_URL` at the appropriate legacy bridge endpoint.
|
||||
|
||||
In the development override, that wiring is already done with:
|
||||
In the current development override, the local legacy handoff wiring is already done with:
|
||||
|
||||
```text
|
||||
FACEAI_LEGACY_RETURN_URL=http://localhost:8080/faceai_return.php
|
||||
FACEAI_HANDOFF_URL=http://localhost:3001/dev/legacy/launch
|
||||
FACEAI_LEGACY_RETURN_MODE=direct
|
||||
FACEAI_LEGACY_HOME_URL=http://localhost:8080/index.jsp
|
||||
```
|
||||
|
||||
|
|
@ -438,18 +448,17 @@ The development override also keeps the log wiring with `./logs:/data/logs`, so
|
|||
|
||||
The Compose contract now also includes an HTTP healthcheck on the public FaceAI service and a Redis readiness check. That makes `docker compose ps` meaningful during rollout: `faceai` only becomes healthy after `GET /health` returns `{"ok":true}`, and both the public site and the processor wait for Redis readiness before their own startup sequence begins.
|
||||
|
||||
The local PHP simulator also needs the legacy bridge feature flag enabled:
|
||||
The local Tomcat stack also needs the legacy bridge feature flag enabled on the JSP side:
|
||||
|
||||
```text
|
||||
FACEAI_FEATURE_ENABLED=1
|
||||
```
|
||||
|
||||
The checked-in `docker-compose.override.yml` sets that on the `legacy-php` service so the simulator can launch the FaceAI handoff flow locally.
|
||||
The checked-in `docker-compose.override.yml` sets that on the local `tomcat-www` service so the race page can launch the FaceAI handoff flow locally.
|
||||
|
||||
## Notes
|
||||
|
||||
- Search orchestration now uses Redis and a dedicated processor worker.
|
||||
- The checked-in base Compose file is production-oriented, while the checked-in override is development-only.
|
||||
- The local legacy simulator is intentionally backend-driven so the handoff can be tested without compiling the existing Java application.
|
||||
- `www/faceai_simulator.php` exists only for local testing. It does not replace the actual JSP race page.
|
||||
- The local development flow now uses the actual Tomcat-served JSP race page instead of a separate PHP simulator bridge.
|
||||
- The final legacy integration still needs a real signed identity source and a real return-filter implementation on the old site.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue