Skip to content

Running the engine on a server ​

By default everything runs inside your application: the engine, the documentation index, the local HTTP server and — with a model provider — the API key, on the user's machine. That is the right answer for an internal application on managed PCs. It is the wrong one when:

  • the application is installed at customers: whoever owns the machine can recover the key;
  • it is mobile: a local server in-process is awkward and a key inside the package is extractable;
  • you need quotas, logs, or one index kept up to date in a single place.

The HelpChatAI server is the other half: the same engine, the same page, the same endpoints, serving many clients.

Starting a server ​

HelpChatAIServerApp --key demo-key --provider claude --index help-index.json --port 8080
HelpChatAIServerApp --write-config helpchatai-server.json     # a starting point (the commented one is helpchatai-server.json.sample)

helpchatai-server.json holds everything:

json
{
  "Port": 8080,
  "BindIP": "0.0.0.0",
  "ClientActions": true,
  "Keys": [ "the-key-of-my-application" ],
  "Sessions": { "Storage": "memory", "Folder": "sessions", "IdleMinutes": 120, "Max": 500 },
  "Provider": "claude",
  "Language": "it",
  "DocIndexFile": "help-index.json",
  "DocBaseUrl": "https://help.acme.com/manual/",
  "Claude": { "ApiKey": "sk-ant-...", "Model": "claude-haiku-5-5" },
  "OpenAI": { "ApiKey": "", "Model": "gpt-6-luna" },
  "Gemini": { "ApiKey": "", "Model": "gemini-3.5-flash-lite" }
}

The server refuses to start without at least one application key: an open server spends your model budget for whoever finds it. /health answers with the version, the number of sessions, the provider, the size of the index and the profiles whose engine is running.

Everything it does lives in THelpChatRemoteServer (unit HelpChatAI.Server.Remote), so you can host the same server inside an application you already have instead of running the console one.

Pointing a client at it ​

A configuration change, no code:

json
{
  "Remote": {
    "BaseUrl": "https://help.acme.com",
    "AppKey": "the-key-of-my-application",
    "UserId": "",
    "UseHostChannel": true
  }
}

TVclHelpChatAI and TFmxHelpChatAI read Settings.Remote and build a remote transport instead of the local server. The component, the page, F1, the screen context and the events stay exactly the same; what changes is that this process no longer loads an index, starts a worker pool or holds a model key.

The session opens the first time the user asks for help, not at startup: an application whose help server is down still starts, and is told about it when someone presses F1.

The Profile of the component travels with the session: the server answers as that assistant when its configuration defines it. What stays on the client and does not travel: the actions, the EnabledCommands and the PDF handler — see what is not there yet.

Sessions and conversations ​

A client opens a session with POST /session presenting its application key and, when the application knows it, a UserId. A session is a conversation of the engine:

  • u:<user> when there is a user — so the same person continues where they left off, from any device;
  • s:<session> when there is not.

One engine per profile serves everybody who asks that assistant: the store keys the conversations, the worker pool is shared and bounded, the index is loaded once. The same user talking to two assistants has two conversations, one per engine.

Sessions.Storage decides what happens when the last session on a conversation ends:

StorageBehaviour
memorythe conversation is forgotten
diskit is written to Sessions.Folder and read back when that user returns, server restart included

Only a conversation somebody can come back to is written: one keyed by its user, or by the key your OnAuthenticate gave it. An anonymous one is keyed by its session, which nobody presents twice, so it ends with it whatever the storage is.

Idle sessions are swept after IdleMinutes. The client does not have to care: a session the server expired, or lost in a restart, is opened again by the transport the next time it is needed, and the panel reloads on the new one - also a panel that simply stayed open: the page that is refused asks its host to load it again. With a UserId the conversation is the same as before; an anonymous one starts empty, because its key was the session.

Assistants on a server ​

A server can be every assistant of your applications, the way one application is: its helpchatai-server.json gets the same Profiles section a client configuration has, and a component with Profile = 'Orders' gets the order expert.

json
{
  "Keys": [ "the-key-of-my-application" ],
  "Provider": "claude",
  "DocIndexFile": "help-index.json",
  "Claude": { "ApiKey": "sk-ant-...", "Model": "claude-haiku-5-5" },
  "Profiles": {
    "Orders": {
      "AssistantName": "Giulio",
      "Instructions": "You are the expert of orders and invoices.",
      "DocAreas": [ "Orders" ]
    },
    "Customers": { "DocAreas": [ "Customers" ] }
  }
}

The client sends the Profile of the component when it opens its session. The first session of a profile creates the engine of that profile - the root of the file with the section of the profile on top, as an application reads it: instructions, areas of the manual, models, allowed actions - and every later session of the same profile shares it. The console log says when one starts, and /health lists them.

A profile the file does not define, and a component with no Profile, are answered by the default engine, the root of the file: the same fallback an application has when its configuration has no section for the profile. A client cannot make the server create engines by naming profiles: only the ones of the configuration exist. The answer to POST /session says which profile the server took ("profile", empty for the default), and so does ServerProfile on the remote transport.

With disk storage each assistant keeps its own files: the default one under the names it always had, the others with the profile in front.

A server hosted in a program of yours decides the settings of a profile with OnProfileSettings. The settings arrive as a copy of the default engine's with Profile set; the handler puts the section of the profile on top and accepts it:

pascal
procedure TMyHost.ServerProfileSettings(Sender: TObject; const AProfile: string;
  const ASettings: THelpChatSettings; var AAccepted: Boolean);
begin
  AAccepted := MyProfiles.IndexOf(AProfile) >= 0;   // only the ones you have
  if AAccepted then
    ASettings.LoadFromFile(MyConfigFile);           // root, then Profiles.<AProfile>
end;

FServer.OnProfileSettings := ServerProfileSettings;

Without a handler every session gets the default engine, whatever profile it names. The handler is called once per profile, on a connection thread, under a lock of the server: it must be thread-safe and must not call back into the server.

Who may connect ​

The application key identifies the application, not the user, and grants nothing but the right to ask questions. To decide per user — against your own user store, your own tenants — handle OnAuthenticate on the server object. It is an event, so the handler is a method:

pascal
procedure TMyHost.ServerAuthenticate(Sender: TObject; const AAppKey, AUserId,
  AUserToken: string; var AAllowed: Boolean; var AConversationKey: string);
begin
  AAllowed := MyUsers.CheckToken(AUserId, AUserToken);
  AConversationKey := 'tenant:' + MyUsers.TenantOf(AUserId) + ':' + AUserId;
end;

FServer.OnAuthenticate := ServerAuthenticate;

It is called on a connection thread, possibly several at once: the handler must be thread-safe and must not touch the UI. The client sends Remote.UserToken with Remote.UserId; nothing in the library looks at the token, it is yours to check.

Without OnAuthenticate the user id is taken on trust

With no handler only the application key is checked, so a client holding a valid key gets the conversation of whichever user it names. The server writes a line in its log at start to say so. The console server HelpChatAIServerApp sets no handler: when the conversations are private, host THelpChatRemoteServer in a program of yours and check the token there.

How the page still reaches the application ​

Served from another origin, the panel could no longer ask the application to close it. The transport opens a small listener on 127.0.0.1 — the host channel — and tells the server where it is; Esc in the page goes there. CORS is restricted to the panel origin, the channel has its own secret, and the page is told when there is none (Remote.UseHostChannel is False, or the listener could not be opened): it then asks nothing of a host that is not listening, and the window of the chat is closed the way your application closes it.

That channel is also where file exchange between page and application will live.

Actions on a server ​

The model runs on the server; the actions and the skills of an assistant run where its form is, in the application. Nothing changes in the code that registers them:

  1. the transport declares them to its session - name, description and parameters, what a local model would be told - when the panel is shown and whenever the screen changes, if they changed (POST /api/actions);
  2. the model of that session is offered them, and when it calls one the call waits on the server;
  3. the page, which is polling the reply, is told that a call is waiting and tells the application through the host channel; the transport takes the call (GET /api/actions/pending), runs it on the component - the main thread, the confirmation dialog, OnActionConfirm, exactly as with a local engine - and posts how it went (POST /api/actions/result);
  4. the model reads the outcome and goes on.

The handler never runs on the server. What travels is the declaration, the arguments the model wrote and the text the handler answered.

  • It needs the host channel. With Remote.UseHostChannel False, or a listener that could not be opened, nothing is declared and the assistant only answers.
  • The server may narrow them, or take none. AllowedActions in the server's configuration, at the root or in a profile, keeps only the names it lists - the actions of a skill included: naming a skill there allows nothing, because the skills live in the client. "ClientActions": false in helpchatai-server.json (THelpChatRemoteServer.ClientActions) refuses every declaration, with a line in the log of the client.
  • What a declaration is. At most 64 actions, names of letters, digits, _ and -; only the name, the description and the parameters are kept. It is text written by a client that has an application key, and it goes into every request of that session: it is paid in tokens by the key of the server.
  • Times. A call is seen by the application within one poll of the page (0.8 s). The server waits 75 seconds for the outcome - the 60 the application gives its own host, plus the round trips - and then tells the model the action timed out. A call the application already took may still run after that, or after the reply was stopped: its outcome is then discarded.
  • An action registered while the panel is open reaches the server with the next change of screen, or the next time the panel is shown.
  • A server of before 2.1 knows nothing of this: the client logs a warning and the assistant only answers.

Models, languages and commands on a server ​

The engine of a profile is shared by every session of that profile, so its models are listed to the panel but not switched by a client (/api/assistant/select answers 403), and the same goes for the language: the panel shows no flag, and /api/language/select answers 403 - the server speaks the language of its configuration. The commands a session accepts are the server's default - everything but prompt, run, which would run an action of the client unasked by a model, and check, which would describe the machine the server runs on - and a client cannot widen them, not even from code.

What the server refuses ​

A server on a network is reached by whoever finds it, so a few things are decided before anybody is asked who they are:

  • the size of a request. A body larger than 1 MB - Attachments.MaxSizeKB plus a margin for the request that carries an attachment - is answered 413 from its headers, before a byte of it is read; a request that declares no length (chunked) likewise;
  • the connections served at once: MaxConnections, 256 by default, 0 for no limit;
  • what an error says. An exception while answering goes to OnLog with its class and its message; the client reads a generic sentence, never a path or a file name of the server;
  • keys and tokens are compared in constant time.

A reverse proxy in front of it is still the place for TLS, rate limits and whatever else your network asks for.

Behind a reverse proxy ​

Mount the server at the root of a host or virtual host: the panel page is served from /. Remember that the clients and the page speak HTTP to it, so terminate TLS at the proxy.

What is not there yet ​

A remote engine does less than a local one today, and it is better to know it before choosing:

  • Saving as PDF answers 501. The document would be built where there is no screen to save it from, and the page says so instead of doing nothing. Copying works, because it never leaves the browser.
  • The model and the language cannot be switched from the panel or from code: both are settings of an engine every session of its profile shares.
  • Attachments are the server's. The page posts a document to the server (POST /api/attach), which keeps it with the conversation of that session and sends it to the model: so it is the server's Attachments.Enabled that shows the clip, not the client's, and the document leaves the user's machine for your server before it reaches the model. AttachFile from code does nothing with a remote engine (it returns False). This is what the code does; the pair has not been tried against a real server yet.
  • Without OnAuthenticate the user id is taken on trust: see Who may connect.

Released under the Apache License 2.0