JSON Broker and Connector

The JSON Broker provides a modern, file-based storage solution for InstantObjects using JSON format. It's particularly useful for:

• Prototype and development - Quick setup without database configuration
• REST services integration - Native JSON format for web services
• Data interchange - Easy export/import of business objects
• Small to medium datasets - File-based storage for lightweight applications
• Configuration storage - Persistent application settings and user data

Components

TJSONFilesAccessor

The connection component that manages JSON file access. It acts as a "virtual database" using the file system as storage.

Key Properties:

• RootFolder - Base directory where JSON files are stored. Each business class gets its own subfolder.
• JSONFileFormat - File encoding format:
  • xffUtf8 (default) - UTF-8 encoding for international character support
  • xffIso - ISO-8859-1 encoding for legacy compatibility
• OnCustomLoadJSONFile - Event for custom JSON file loading logic
• OnCustomSaveToJSONFile - Event for custom JSON file saving logic

Example Setup:

var
  JSONAccessor: TJSONFilesAccessor;
begin
  JSONAccessor := TJSONFilesAccessor.Create(Self);
  JSONAccessor.RootFolder := 'C:\MyApp\Data';
  JSONAccessor.JSONFileFormat := xffUtf8;
  JSONAccessor.Connected := True;
end;

TInstantJSONConnector

The connector component that integrates the JSON broker with InstantObjects persistence layer.

Key Properties:

• Connection - Reference to a TJSONFilesAccessor component
• IsDefault - Set to True to make this the default connector for the application
• BrokerClass - Automatically set to TInstantJSONBroker

Example Setup:

var
  JSONConnector: TInstantJSONConnector;
begin
  JSONConnector := TInstantJSONConnector.Create(Self);
  JSONConnector.Connection := JSONAccessor;  // Previously created TJSONFilesAccessor
  JSONConnector.IsDefault := True;
end;

File Organization

The JSON broker organizes files by class name:

RootFolder/
├── TContact/
│   ├── CONT001.json
│   ├── CONT002.json
│   └── CONT003.json
├── TCompany/
│   ├── COMP001.json
│   └── COMP002.json
└── TCategory/
    ├── CAT001.json
    └── CAT002.json

Each JSON file contains:

• Object class name
• Object ID
• Update count (for optimistic locking)
• All attribute values
• Nested objects (for Part attributes)
• References (for Reference/References attributes)

JSON File Format Example

{
  "ClassName": "TContact",
  "Id": "CONT001",
  "_Name": "John Smith",
  "_Email": "john.smith@example.com",
  "_Category": {
    "ReferenceObject": "CAT001"
  },
  "_Phones": [
    {
      "_Number": "+1234567890",
      "_PhoneType": 0
    }
  ]
}

Features and Capabilities

Supported Operations

• CRUD operations - Create, Retrieve, Update, Delete
• Query support - Limited query capabilities (class-based retrieval)
• Optimistic locking - Update count tracking for concurrency control
• Blob support - Binary data encoded in JSON
• Relationship support - Part, Parts, Reference, and References attributes

Limitations

• No complex queries - Cannot perform SQL-like queries with JOINs or complex WHERE clauses
• Performance - Not suitable for large datasets (thousands of objects)
• No transactions - Each operation is atomic but multi-object transactions are not supported
• Limited indexing - No database-level indexing for fast retrieval

Use Cases

1. Rapid Prototyping

// No database setup required!
var
  Contact: TContact;
begin
  Contact := TContact.Create;
  try
    Contact.Name := 'John Doe';
    Contact.Email := 'john@example.com';
    Contact.Store;  // Automatically saved to JSON file
  finally
    Contact.Free;
  end;
end;

2. Configuration and Settings

// Application settings as persistent objects
TAppSettings = class(TInstantObject)
  _Theme: TInstantString;
  _Language: TInstantString;
  _AutoSave: TInstantBoolean;
end;

// Load settings
var
  Settings: TAppSettings;
begin
  Settings := TAppSettings.Retrieve('DEFAULT', True);  // Create if missing
  try
    ApplyTheme(Settings.Theme);
    SetLanguage(Settings.Language);
  finally
    Settings.Free;
  end;
end;

3. Data Export/Import

// Export objects to JSON for backup or transfer
var
  Contact: TContact;
  Contacts: TObjectList;
begin
  Contacts := TObjectList.Create(False);
  try
    // Retrieve all contacts
    TContact.RetrieveAll(Contacts);

    // They're already in JSON format in the files!
    // Just copy the entire folder for backup
    CopyDirectory(JSONAccessor.RootFolder, BackupPath);
  finally
    Contacts.Free;
  end;
end;

Integration with Neon Serialization

The JSON broker can be enhanced with the Neon serialization library for more advanced JSON features:

• Custom property names
• Property filtering
• Complex type support
• JSON Schema validation

See JSON Serialization with Neon for details.

Best Practices

1. Use appropriate RootFolder - Choose a location with proper write permissions
2. Backup regularly - JSON files are just files - use standard backup procedures
3. Monitor file count - Performance degrades with thousands of files per folder
4. Use with small datasets - Consider SQL brokers for larger applications
5. Version control friendly - JSON format works well with Git and other VCS
6. Test concurrency - Update count mechanism prevents lost updates but doesn't lock files

Migration from/to SQL Brokers

From SQL to JSON

// Use TInstantPump to copy data between brokers
var
  Pump: TInstantPump;
begin
  Pump := TInstantPump.Create(nil);
  try
    Pump.SourceConnector := SQLConnector;
    Pump.DestinationConnector := JSONConnector;
    Pump.Execute;
  finally
    Pump.Free;
  end;
end;

From JSON to SQL

// Reverse the connectors
Pump.SourceConnector := JSONConnector;
Pump.DestinationConnector := SQLConnector;
Pump.Execute;

Troubleshooting

Problem: "Access denied" error when saving

• Solution: Check write permissions on RootFolder

Problem: Files not appearing

• Solution: Verify RootFolder path and ensure folder exists

Problem: Corrupt JSON files

• Solution: JSON files are human-readable - open with text editor to inspect and manually fix if needed

Problem: Poor performance with many objects

• Solution: Consider migrating to a SQL-based broker (FireDAC recommended)

See Also

• The Connector - Overview of all brokers
• JSON Serialization with Neon - Advanced JSON features
• WiRL REST Server - Using JSON with WiRL REST services
• MARS Curiosity REST Server - Using JSON with MARS REST services
• External Storage - External attribute storage