User Guide

Logging in

Open the service's URL in a browser and log in with your email address and password. There's no self-service signup -- an Admin creates your account (see Admin: Users).

Three roles:

Admin -- everything, including user/config management.
Consultant -- compile/edit maps, launch/view runs.
Viewer -- view maps and run history (launching is configurable per install).

Roles and AI-analysis access are two separate things -- see AI Analysis below.

Maps

A map defines how an input file's columns become records loaded into QAD (via REST, QXtend, or written out as a formatted Excel file). Maps are defined in a fixed-layout Excel workbook and compiled into a versioned, immutable map.

Go to Maps, give the map a name, and upload the workbook. A compile error shows a full list of problems (missing columns, bad formulas, unresolved xref tabs, ...) so you can fix the workbook and re-upload instead of stopping at the first issue.
A successful compile creates version 1 of that map. Uploading the same map name again always creates the next version (v2, v3, ...) -- a map version is never overwritten, so every run stays reproducible against the exact map it used.
Open a map version's detail page to review the resolved Source contract, Target adapter settings, and every Field Map rule.

Round-trip mode: extraction-sourced xref. A Field Map xref rule normally sources its lookup table from an in-workbook xref tab (Xref Tab / Xref Key Column / Xref Value Column). It can instead source that table from an extraction profile's latest successful normalized dataset by filling in Extraction Profile Family / Extraction Dataset / Extraction Key Column / Extraction Value Column and leaving the Xref Tab columns blank (the two groups are mutually exclusive -- compilation fails if both or neither are set). The lookup table is resolved and baked into the compiled map at compile time, from whichever extraction for that profile family most recently succeeded -- run the extraction at least once before compiling, and recompile the map to pick up newer extraction data. See docs/specs/2026-07-11-round-trip-xref-design.md for the design.

Data-quality Rules tab. A map's workbook can include an optional Rules tab declaring per-field data-quality checks against already-mapped target-field values. Each row is one rule:

Column	Meaning
Rule Name	Optional label; defaults to Target Field if left blank.
Target Field	Required -- must be a field produced by a Field Map rule in the same workbook.
Row Type	Optional -- restricts the check to records of that row type only (blank = all rows).
Check Type	`range`, `format`, or `referential`.
On Fail	`warn` (default) or `drop`.
Min Value / Max Value	`range` checks -- at least one of the two is required.
Pattern	`format` checks -- a regex the value must fully match.
Extraction Profile Family / Extraction Dataset / Lookup Column	`referential` checks -- the value must appear in that extraction's latest successful normalized dataset (same "resolved and baked in at compile time" mechanism as round-trip xref above -- run the extraction at least once before compiling, and recompile to pick up newer data).
Message	Optional custom finding text; otherwise a default message is generated.

Every run of a map with a Rules tab (dry or real) evaluates all its rules right after mapping and writes a Data-quality rules section on the run detail page (plus rules_report.md/ rules_report.json in the run folder) -- a warn rule only reports; a drop rule removes the failing row via the map's existing failure policy, exactly as a transform-stage error would. This runs unconditionally for any map that has a Rules tab -- there is no admin-level on/off switch for it (unlike AI address analysis's Enable AI address analysis toggle under Admin: System Configuration). A map with no Rules tab shows no such section and writes no rules_report.* files.

The Rules tab is a completely separate mechanism from the Field Map tab's Field Type: address column described under AI address analysis below -- one declares informational formatting/ duplicate-detection suggestions on address fields, the other declares field-level pass/fail checks (with an optional row-dropping consequence) on any field. A map can use either, both, or neither.

Editing a map in the browser. Admins and Consultants can open a map version's detail page and click Edit this version (/maps/{name}/v{version}/edit) instead of re-uploading a whole workbook to change one field. The editor is a tabbed page -- Target, Source, Field Map, Row Types, Xrefs -- where every row of every tab is add/remove/edit-able directly in the browser; the Xrefs tab also accepts a pasted CSV/TSV block to replace an entire xref table's rows in one paste instead of editing rows one at a time. Clicking Save & Recompile re-validates everything exactly like a workbook compile (the same error list, same all-tabs- preserved re-render on failure) and, on success, always creates a new map version -- editing never modifies an existing version or touches the .xlsx file on disk; the original uploaded workbook is untouched and never regenerated from your edits.

The editor does not yet expose a Rules tab: saving an edit always produces a new version with an empty Rules list, even if the version you started editing had rules. If a map has a Rules tab, edit it by re-uploading a workbook (via Maps > upload) that includes the Rules tab you want, not via Edit this version.

Because a browser-edited version no longer matches any workbook file, it's flagged with a drift badge/banner (on /maps and on that version's detail page) so it's obvious at a glance that re-uploading the original workbook would not include those edits -- the workbook and the live map have diverged. The only way to clear drift is to upload a fresh workbook (via /maps/compile) that already contains the edits you want; the resulting new version is workbook-sourced again and the badge disappears.

Runs

Go to Runs > Launch, pick a map + version, upload the input file, and choose whether to dry-run (parses/transforms/validates everything except the final load -- safe to use against production QAD).
The run queues and executes on a single sequential worker shared by every trigger (web UI, CLI, REST API, and watch-folder) -- launching several runs at once just queues them in order rather than racing each other.
The run detail page shows live status, then per-record results, any parse/transform issues, and (for REST/QXtend targets) a downloadable re-runnable file of any rows that failed under a skip-and-continue or threshold failure policy.

Dry-run preview. Every dry-run additionally produces a preview report: a client-readable preview_report.md summary (what would be loaded/dropped, issue-frequency tables, top failed-row samples) shown inline on the run detail page, plus a full analyst-detail preview_records.xlsx workbook (side-by-side source-to-target columns for every mapped record, a Dropped Rows sheet, and an Issues sheet). Both are downloadable from the run detail page. No artificial row cap -- large dry-runs (above 50,000 mapped records) still produce the full workbook, just without header styling.

AI address analysis. If a map flags one or more target fields as address fields (a "Field Type: address" column value on the Field Map tab -- see your map author for how a given map is set up), every run of that map, dry or real, produces an AI Address Analysis section on the run detail page: normalized address formatting suggestions and likely-duplicate row clusters found within that run's input. This is informational only -- it never blocks or changes the load, and nothing is auto-applied to the loaded data; a human reviews the suggestions and clusters and decides what (if anything) to do about them. It requires both the map's own opt-in and an Admin having turned the feature on system-wide (see Admin: System Configuration); if either is off, the run behaves exactly as if the feature didn't exist -- no section, no address_analysis.* files written.

AI Analysis

Every completed non-Excel run always gets a deterministic pre-check (loaded vs. attempted counts, per-record status, error-code frequency) with no AI involved.

On top of that, a run detail page can show an Analyze with AI button that drives an LLM-backed, tool-calling analysis: the browser itself calls the configured LLM directly (never proxied through the server) and can query the QAD database (via a server-side MCP proxy) to double-check that what was reported as loaded is actually correct there. The full exchange (everything the AI asked and got back) is saved to the run folder.

This is also available for dry-runs: the AI is given the deterministic dry-run preview (parse/ transform issue counts and error-frequency table) instead of a load precheck, so an analyst can get an AI read on a preview before committing to a real load.

This button only appears when both of the following are true:

An Admin has configured the shared LLM setup at LLM Configuration (/admin/llm-config) -- one setup for the whole system, not per-analyst.
Your account has been granted AI-analysis permission by an Admin (see Admin: Users) -- this is independent of role; even an Admin account doesn't have it by default except the very first bootstrap account.

Admin: Users

/admin/users (Admin only).

Add a single user: email, password, role, and whether to grant AI-analysis permission up front.
Toggle AI-analysis permission per existing account at any time via the checkbox in the table -- off by default for everyone except the bootstrap admin.
Delete a user.
Bulk import via Excel: click Download import template (.xlsx) to get a workbook with Email, Role, Can Run AI Analysis, and Can Use Prophet's Vision columns -- Role, and the two permission columns, each have an Excel dropdown (Role restricted to valid roles; the permission columns to TRUE/FALSE) so a typo can't slip through. The two permission columns are optional -- leaving them blank grants neither permission, same off-by-default posture as the single-user "Add user" form. Fill in one row per person, then upload it under Bulk import users. Each new account gets a random temporary password, shown once in the results table right after import (there's no automated email delivery yet -- pass the password along to the new user yourself). Rows with a bad email, an invalid role, or a duplicate email are reported individually and skipped without failing the rest of the batch.

There's currently no self-service "forgot password" or admin-triggered password reset from the web UI -- to change a password today, delete and recreate the account (bulk re-import works fine for this too).

Admin: LLM Configuration

/admin/llm-config (Admin only). One shared setup for the whole system: base URL, API key, model name, and whether the model supports tool-calling. The API key is never re-rendered into the page after saving -- leave it blank on a re-save to keep the existing key.

Admin: System Configuration

/admin/system-config (Admin only). QAD REST and QXtend connection settings, the MCP server address, max concurrency, and the REST API's shared token. Unlike config.yaml, changes here take effect immediately -- no restart. See docs/CONFIGURATION.md in the repo for exactly which settings live here versus which ones still require editing config.yaml and a restart (storage paths, host/port, the session secret, watch-folder mappings).

Also here, under Data quality: Enable AI address analysis, the system-wide switch for the AI address normalization + duplicate-detection feature described under Runs. Off by default -- a map's own "Field Type: address" opt-in has no effect until an Admin also turns this on.

Extractions

An extraction pulls data off a source system that has no REST/QXtend endpoint for it -- an older QAD/Progress database, reachable only over SSH -- instead of a file someone uploads. It follows a compiled extraction profile (probe the host, optionally push and run a script, fetch the result, normalize it, then deliver it) and lands its output either straight in a map's watch-folder inbox or as an ordinary run, exactly like any other trigger.

Go to Extractions > Launch, pick a profile family + version, optionally a client (selects that client's overlay -- see below), and which source connection to run against.
Check Probe only to just run the profile's fingerprint checks (confirms the host matches what this profile version expects) without pushing, executing, or fetching anything -- useful for confirming a new source connection or a version-drift concern before committing to a full run.
The extraction detail page shows each probe's pass/fail result, and (for a full run) the push/execute/fetch steps and where the normalized data was delivered.

Compiling a profile or overlay bundle

Extraction profiles are versioned and compiled the same way maps are -- a source bundle (a profile.yaml, plus an optional scripts/ directory of files it pushes to the source host) is validated and turned into an immutable, new version. Go to Extract Profiles > Upload / compile a profile bundle, or use the CLI:

etlp profile compile ./my-profile-bundle --family my_family

A client overlay layers one client's deltas on top of a shared base profile (e.g. a renamed field or a different default parameter) without duplicating the whole profile -- compile it from the family's page (compile overlay) or via etlp profile overlay-compile. See docs/examples/extract_profiles/ in the repo for two complete, ready-to-compile example bundles (a simple flat-file drop, and a full push+execute+probe QAD/Progress profile with a client overlay) and their own README explaining what each demonstrates.

Admin: Source Connections

/admin/source-connections (Admin only). Each row is one SSH destination an extraction profile can run against: host, port, username, auth method (key or password, via the usual *_env_var indirection -- no secret is ever entered directly), and connect timeout.

New connections are not trusted automatically. Click Test Connection to connect far enough to read the host's SSH host key (without authenticating) and pin its fingerprint on that row -- every later connection through it is verified against that pinned fingerprint, so a host key that changes later is treated as a failure rather than silently re-trusted.

Admin: Scheduled Extraction Runs

/admin/schedules (Admin only). A schedule fires an extraction automatically on a fixed "every N minutes" cadence, instead of someone clicking Launch by hand each time.

To create one, pick a profile family + version, an optional client (selects that client's overlay, same as a manual launch), which source connection to run against, and how often (in minutes) it should fire. The schedule fires on its own -- no need to also visit Extractions -- and each run it kicks off shows up in the Extractions list with trigger_type = scheduled, exactly like any other extraction: probes run, data is fetched and normalized, and a successful delivery auto-chains into a load run just as a manually-launched extraction's would.

A schedule whose previous run is still in progress is skipped rather than piled up behind it -- the next check is simply pushed out to the schedule's own cadence, so a slow extraction can't cause a backlog of duplicate runs.

Pause stops further firing without deleting the schedule or its history (its last_run_at/ next_run_at stay visible); Resume picks firing back up. Delete removes the schedule entirely (past extraction rows it already fired are untouched). Edit a schedule's family, version, client overlay, connection, or interval inline from its row.

This is an interval, not a fixed clock time -- "every 60 minutes" from whenever it was created/last fired, not "at 2:00am daily."

Dataset Comparisons

A comparison diffs two datasets by one or more key columns and reports what's added, removed, and changed between them (plus how many rows were unchanged). Each side (left and right) is either an existing extraction's normalized dataset or a file you upload on the spot. Three common use cases:

Env-vs-env: the same extraction dataset pulled from two source connections (e.g. TEST vs. PROD) -- confirms two environments actually hold the same data.
Run-vs-run: the same extraction profile run at two different times -- shows what changed between runs.
Pre/post load check: a source file compared against the corresponding extraction dataset after a load, to confirm the load didn't silently drop or alter records.

Go to Comparisons > Launch a comparison. For each side, pick Extraction dataset (an extraction id + dataset name) or Upload a file.
Enter key column(s) (comma-separated for a composite key, e.g. pt_part,pt_site) -- or leave blank when both sides deliver to the same compiled map, so the map's own key column is used as the default.
Optional settings: ignore columns (comma-separated columns excluded from change detection, e.g. timestamps), case-insensitive/whitespace-trimmed value matching, and how to handle a duplicate key or a column-set mismatch (fail the comparison, or warn and compare what's shared).
The comparison detail page shows summary counts, any warnings, a table of changed records (key, column, old value, new value), and a Download differences.csv link for the full result.

A comparison that hits a genuine data problem (a missing source file, a duplicate key when set to fail, a column-set mismatch when set to fail) lands with status = failed and a specific error message -- it never silently produces a partial diff.

QAD Build Order

The Build Order page is an editable implementation checklist of every QAD Adaptive configuration menu -- setup screens, master data, and control files -- sequenced in the order you'd configure them. It ships pre-loaded with 546 rows extracted from the QAD Online Help (v2022 EE), and each row carries a Build Order code like A01.005:

The letter is the module, in dependency order (A = Base Data, B = Financials, ... O = Administration).
The two digits are the functional group within the module (e.g. GL Setup, Item and Site Data).
The three digits after the period are the sequence, stepped by 5 so you can insert new rows between existing ones (e.g. A01.007 slots between A01.005 and A01.010).

Everyone who can log in can view, filter, and export the list. Only an Admin can edit it.

Working the list in the browser (Admin):

Filter by Module, Group, or Status, or search across codes, menu names, owners, and notes.
Click a cell -- Status, Owner, Config Date, Notes, Description, Prerequisites, Menu Name, or Page -- to edit it in place. Enter saves, Escape cancels. Status is a dropdown (Not Started / In Progress / Complete / N/A) and shows as a colored badge.
Add Row creates a new entry under any unused code; the x button on a row deletes it.

Excel round-trip:

Export to Excel downloads the whole list (with a Status dropdown baked into the file).
Edit it in Excel -- update statuses/owners/notes, add rows with new codes, delete rows.
Import from Excel (Admin) uploads it back. Rows are matched by Build Order code: existing codes are updated, new codes are inserted, and rows that exist in the app but are missing from your file are flagged in a report -- never deleted automatically (each flagged row gets its own Delete button so removals stay deliberate).

An import is all-or-nothing: if any row has a malformed code or invalid value, the whole file is rejected with row-numbered errors and nothing is changed. The Help Page column refers to the page number in the QAD Online Help document the menu was extracted from.

CLI

The etlp command talks to the REST API front door (token-authenticated, separate from the web UI's per-user login) -- useful for scripting or CI:

etlp compile --map-name customers --workbook customers.xlsx --base-url http://host:8000 --token <api_token>
etlp run --map customers@v3 --input file.csv --dry-run --base-url http://host:8000 --token <api_token>
etlp status <run-id> --base-url http://host:8000 --token <api_token>

etlp compare submit --left-extraction <id1> --left-dataset item_master \
  --right-extraction <id2> --right-dataset item_master --key id \
  --base-url http://host:8000 --token <api_token>
etlp compare status <comparison-id> --base-url http://host:8000 --token <api_token>

Watch-folder

Each map name listed in config.yaml's watch_folder section gets its own inbox/<map>/ directory (under storage.data_dir). Drop a file in there and it's picked up automatically, launched against the pinned map version, and moved to processed/ (with a report) or rejected/ depending on the outcome. Watch-folder mappings require a restart to change -- see docs/CONFIGURATION.md in the repo.