Canonical TSAPI series DB schema v1.0.0
Single source of truth for the timeseries PostgreSQL schema shared by the Python (TSAPI_SERIESDB_PG_PY) and C++ (TSAPI_POSTGRES_CPP) libraries. SQL seeded byte-for-byte from TSAPI_SERIESDB_PG_PY/tsapi_pgdb/sql on 2026-09-02. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
commit
eed06559c0
17 changed files with 2080 additions and 0 deletions
104
README.md
Normal file
104
README.md
Normal file
|
|
@ -0,0 +1,104 @@
|
|||
# TSAPI_SERIESDB_SCHEMA — canonical timeseries DB schema
|
||||
|
||||
**Single source of truth** for the PostgreSQL schema used by the TSAPI
|
||||
timeseries database. Both client libraries target the schema defined here:
|
||||
|
||||
- **Python** — `TSAPI_SERIESDB_PG_PY`
|
||||
- **C++** — `TSAPI_POSTGRES_CPP`
|
||||
|
||||
Neither project should keep its own copy of the DDL. When the schema changes,
|
||||
it changes **here**, and both projects re-provision / re-test against it.
|
||||
|
||||
Current schema version: **1.0.0** (see [`VERSION`](VERSION); recorded at runtime
|
||||
in the `meta.schema_version` table).
|
||||
|
||||
---
|
||||
|
||||
## Layout
|
||||
|
||||
```
|
||||
TSAPI_SERIESDB_SCHEMA/
|
||||
├── VERSION # canonical schema version string
|
||||
├── run_setup.py # provisioner — runs the numbered files in order
|
||||
├── db_config.json.example # copy to db_config.json and edit (gitignored)
|
||||
├── README.md
|
||||
└── sql/
|
||||
├── 00_extensions.sql # pg_trgm
|
||||
├── 01_schemas.sql # meta / series / log schemas
|
||||
├── 02_sequences.sql # id sequences
|
||||
├── 03_tables.sql # meta.* and log.* tables
|
||||
├── 04_indexes.sql # indexes + registry constraints
|
||||
├── 05_procedures.sql # meta.* / log.* stored procedures
|
||||
├── 06_triggers.sql # prevent_series_drops event trigger
|
||||
├── 07_reference_data.sql # timezones + currencies reference data
|
||||
├── 08_script_log.sql # auxiliary (not part of a fresh install)
|
||||
└── migrate/ # one-off migration scripts (run manually)
|
||||
```
|
||||
|
||||
`run_setup.py` executes `00`–`07` in order. `08_script_log.sql` and
|
||||
`sql/migrate/*.sql` are auxiliary and are **not** run by a fresh install.
|
||||
|
||||
---
|
||||
|
||||
## Provisioning a database
|
||||
|
||||
```bash
|
||||
# 1. configure
|
||||
cp db_config.json.example db_config.json # then edit host/user/password
|
||||
# (or pass connection flags on the command line instead)
|
||||
|
||||
# 2. create the schema
|
||||
python run_setup.py # uses db_config.json / defaults
|
||||
python run_setup.py --host 192.168.1.7 --user claude --password claudepwd
|
||||
python run_setup.py --drop-existing # drop meta+series+log first
|
||||
python run_setup.py --clean-slate # drop ALL user objects first
|
||||
```
|
||||
|
||||
Requires `psycopg2` and a PostgreSQL user with CREATE privileges on the target
|
||||
database.
|
||||
|
||||
---
|
||||
|
||||
## What gets created
|
||||
|
||||
- **Schemas:** `meta` (registry + reference data), `series` (timeseries data
|
||||
tables live here), `log` (event/audit trail).
|
||||
- **Registry:** `meta.series` — one row per series table, PK `table_id`,
|
||||
`UNIQUE (db_schema, table_name)`.
|
||||
- **Reference tables:** `meta.currencies`, `meta.timezones`, `meta.exchanges`,
|
||||
`meta.instruments`, `meta.sessions`, `meta.holidays`, `meta.security_types`,
|
||||
`meta.schema_version`.
|
||||
- **Procedures:** `meta.drop_series` / `_if_exists` / `drop_all_series`,
|
||||
`meta.update_series_last_timestamp`, `meta.refresh_series_timestamps`,
|
||||
`meta.read_series_batch[_cursor|_ohlc]`, `log.notify_event_log`,
|
||||
`log.flush_pending_to_event_log`.
|
||||
- **Event trigger:** `prevent_series_drops` — blocks direct `DROP TABLE` on a
|
||||
registered series (drops must go through `meta.drop_series*`).
|
||||
|
||||
Series data tables have **no `UNIQUE` constraint on `timestamp`** — the index is
|
||||
BRIN. Concurrent writers must serialise per-series with the advisory-lock
|
||||
protocol (`pg_advisory_lock(hashtext('tsapi_pgdb.write:{schema}.{table}'))`); see
|
||||
the C++ project's `planning/2026-09-02_cpp_locking_interop_plan.md`.
|
||||
|
||||
---
|
||||
|
||||
## Versioning contract
|
||||
|
||||
Bump [`VERSION`](VERSION) whenever `sql/` changes in a way clients must know
|
||||
about, and have each client read `meta.schema_version` on connect and warn/fail
|
||||
on mismatch. This turns silent schema drift into a loud error.
|
||||
|
||||
---
|
||||
|
||||
## Provenance
|
||||
|
||||
The `sql/` files were seeded (byte-for-byte, 2026-09-02) from the then-current
|
||||
authoritative copy in `TSAPI_SERIESDB_PG_PY/tsapi_pgdb/sql/`. From this point on,
|
||||
**this** directory is authoritative and the Python project should consume it
|
||||
rather than keep its own copy — see the migration note in the C++ project's
|
||||
`planning/` folder.
|
||||
|
||||
> **Known drift caught during setup:** on 2026-09-02 the remote instance
|
||||
> `192.168.1.7` had `meta.sessions.exchange_id` as nullable, while the canonical
|
||||
> DDL (`03_tables.sql`) defines it `NOT NULL`. Re-provision drifted databases
|
||||
> from these files.
|
||||
Loading…
Add table
Add a link
Reference in a new issue