A flight with a stop saves a seat per leg, but every view read the
booking's flat seat, which mirrors the first leg's. A seat booked only on
the second leg showed nowhere, and one on the first leg showed on both.
#2701 had fixed this for trains only.
- getFlightLegs reads each leg's seat. The first leg falls back to the flat
seat, as the booking form does, so older bookings keep theirs.
- bookingSeats lists the seat of every leg that has one with its route.
The detail views (the phone's transport sheet, the booking detail) show
each leg's seat under its route ("Seat IST → JFK"), and a leg opened
from its row shows its own. The cards and field grids (bookings list,
the phone's Bookings and Transports tabs, the day plan's transport
detail) hold every seat in one Seat cell, so a card cutting its grid
short no longer drops the second leg's.
- The day plan's flight leg rows carry their seat, and the PDF prints a
flight's seat, direct or per leg.
- The watch and the Wallet pass name every leg's seat of the booking.
- A legs-only MCP update sets the flat seat, airline and flight number
(train number and platform) again from the new first leg, so a seat
taken off the first leg does not live on in the flat seat. A flat value
the stored first leg never had stays.
Closes#3158
The second run of #3157 was red in six jobs:
- Server types: ReminderJobsService takes AddonsService now; RJOB-007 built
it with five arguments.
- Server tests 2/10: the doc-sync job tests stubbed AddonsService without
isEnabledForTrip, which the sync asks per trip now.
- Client tests 4/10: toursPermissions mocked useAuthStore as a bare
selector; tripAddonsStore keys its lists by account through getState.
- Server lint (knip): TripAddonRow was exported without a reader outside
its file.
- Client lint and Client types: no-unused-vars and the strict errors in
useTripPlanner.ts went down by one, so their entries are lowered.
A trip can now turn a trip addon (lists, costs, documents, collab, road
trip, tours) on or off for itself, whatever the instance switch says. No
row in the new trip_addon_overrides table means the trip follows the
instance; a row is the owner's choice. Going back to the instance setting
deletes the row, and no addon data is touched either way.
- GET/PUT /api/trips/:tripId/addons[/:addonId] (every member reads, the
owner writes) and the MCP tools get_trip_addons and set_trip_addon, on
the shared tripAddons contract; tripAddons:changed tells the members.
- Every trip-scoped addon surface asks the trip: REST through
TripAddonGuard, MCP through a per-call gate on the named or pinned trip,
plugin RPC and events, the share link, the offline bundle, reminders,
doc sync, the assistant (tools, keep_attached_file), booking costs, the
trip summary prompt, and the cross-trip tour reads (library, Atlas
stats, collection save).
- Routes without a trip open while the addon is on in any of the user's
trips. Switching an addon on that the instance has off ends the members'
MCP sessions so they reconnect with its tools.
- A copied trip takes its owner's choices along.
Behaviour changes: an anonymous call to an addon route that is off now
gets 401 instead of 404, and with an instance switch off the packing,
todo and files plugin RPC methods, the files MCP tools, todo reminder
mails, core plugin events, plugin view contributions, the share link's
lists, costs and chat sections and the offline bundle's lists follow it.
Server Postgres 18 3/3 was red on dev: PGMOVENUL-007 failed with relation "information_schema_catalog_name" does not exist. readTargetSchema called pg_get_serial_sequence(format('%I', table_name), ...) over information_schema.columns, and Postgres does not promise to apply the table_schema filter first, so the call could see rows of information_schema itself, whose bare names do not resolve on the search path. The name now carries its schema, so the call answers for every row whatever the plan. The fake Postgres of the move tests matches the new text.
The 5.0.0 backfill (Migration20200101042800) set email_self_asserted only for accounts whose address moved away from their user.register audit row. An account created in Admin > Users has only an admin.user_create row, so it kept 0 even when the account had changed its own address in its settings before the upgrade, and a verified SSO sign-in asserting that address was still linked into it.
A new migration applies the same rule with the address in the admin.user_create row as the baseline: an admin-made account whose address is none it was created or registered with, and whose email no admin edited, is marked. Only rows still at 0 are touched and nothing is cleared. The shipped migration stays as it is.
The calendar feed and the .ics export wrote a restaurant booking linked to a
plan stop twice, and the booking's copy without a time zone:
- The zone of a booking came only from its own place. A booking whose place
is its linked stop (reservations.assignment_id) had none, so its DTSTART and
DTEND went out as floating times and calendar apps placed them in the
device's zone, an hour early for a London lunch read in Paris. The zone now
comes from the booking's place, then its linked stop's place, then the zone
all stops of its day or of the trip agree on. Transports keep their
endpoint zones and still float when those are unknown, since either end can
be in another zone.
- The stop and its booking were both emitted (trek-assign-<id> and
trek-res-<id>). As for hotels, a stop whose booking yields a timed event on
the same day is no longer emitted on its own; the booking event takes over
the stop's place as LOCATION, its notes and, when it has none, its end. A
booking without a clock, on another day, or still staged leaves the stop as
it was, so no event is lost.
The VTIMEZONE builder moved unchanged into calendar-timezones.helpers.ts to
keep the service under the size cap.
Closes#3150
files, security, collections, journey and trips cleaned up by removing a whole
uploads category folder (uploads/files, covers, places, journey). Every suite
that boots the app writes into the same uploads tree, and on the Postgres lane
several run at once, so that removal took the files of the suite running
beside it: ATTE2E-008 then failed with "storage object not found" for its
attached file.
A small helper wraps StorageService.put on the app's instance, records what
the suite stores and deletes exactly that afterwards. The journey upload tests
that counted the files in uploads/journey (and so also counted what another
suite wrote there) now check that every object they stored is gone again.
Public trip and journey share links handed out more than the owner chose to
share:
- A travel-only trip link still returned every day's title, and every link with
the map returned the free-text day notes the shared page never shows. Days now
go out through a public projection without notes, and travel-only links carry
no day titles. Day notes, packing, budget and chat use explicit field lists
too, and the category list holds only the categories the shared places use,
without creator ids.
- The public journey payload spread whole entry and photo rows, including
storage keys (file_path, thumbnail_path) and internal fields. It is now built
from allow-lists and pinned by a strict response contract.
- /uploads/journey served every entry photo, original and video to anyone who
knew the file name, so a URL kept working after a share link was revoked or
the gallery switched off. It now serves journey covers and other non-entry
objects only; entry media go through the token-checked proxy.
- Anonymous byte paths served stored originals unchanged, so EXIF GPS left the
server even with map sharing off. Every public path (journey share proxy,
covers, trip share photos, public uploads) now sends a copy with location
data blanked in place (EXIF GPS, XMP and IPTC location, MP4/MOV location
atoms), falling back to a metadata-free re-encode or a 404, never the raw
file. Owners and members still get their originals.
- An entry's visibility field never decided the share link but was offered by
MCP and the assistant as if it did. It is now documented as having no effect
and no longer shown as a privacy label; is_draft keeps an entry off the link.
The Atlas page showed 100 cities and 169 places where GET /api/v1/stats
and the dashboard's passport card showed 66 and 164 for the same account.
The two sides counted with their own code. The Atlas page took the address
part right before the country as the city, which is often the region
(Bavaria for Munich, Nuremberg and Regensburg alike), and counted a bare
place name with no comma as a city. The stats side read its places with a
DISTINCT over address, coordinates and region only, so two places that
looked alike (the same hotel on two trips, places with neither address nor
coordinates) counted once, and its city reader skipped every address part
with a digit, so "10178 Berlin" never counted as Berlin.
Both now take places and cities from one method: every place counts once,
a city is read with cityFromAddress, which skips the region (#1115) and now
reads a city written beside its postcode (German, French, US, Japanese,
Dutch, British and Canadian forms) while a street with its house number
still does not count, and two spellings that differ only in case are one
city. The Atlas page, the passport card, /api/v1/stats and the MCP atlas
tool follow it. The response shapes are unchanged.
Closes#3137
Started as root, the entrypoint dropped to the node user with gosu. gosu is
a Go program, and Go changes a process's ids by sending every runtime thread
a realtime signal and spinning until each one has answered, with no limit.
On a host whose security module drops that signal inside the container
(AppArmor on recent Ubuntu, under rootful Podman) gosu spun at full CPU
forever, before Node printed a line: no logs, no database, port 3000 never
served. Running as uid 1000 worked only because that path skips gosu.
The entrypoint now drops privileges with setpriv from util-linux, which is
already in the image: single-threaded C that makes the same id changes
without any signal. Node still runs as uid and gid 1000 with no
capabilities and HOME=/home/node. gosu stays in the image for anyone who
calls it by hand.
Closes#3135
The v1 bodies were cast to their types unchecked, so a flight page whose data
was not a list, or a passenger list of another shape, failed deeper down as a
TypeError instead of an AirTrail request error. Each v1 answer (a flight, a
page of flights, a created flight, a reference lookup, the key owner) is now
read through a small schema that keeps the fields TREK does not need lenient.
The legacy save refuses an arrival before the departure; the v1 body passed
it on, so an overnight arrival time without its date would have been stored
before the departure. The v1 write-back now refuses it with the same message.
AirTrail 3.13 adds /api/v1 and marks the /api/flight/* routes as
deprecated. TREK now asks the public discovery document GET /api/v1
and uses v1 when the instance has it. It uses the legacy routes for
an older instance. The client methods keep their signatures, and the
sync hash stays the same, so a linked flight does not sync again
when an instance changes to v1.
The probe cache holds only a clear answer: 200 with the discovery
document gives v1, and 404 gives legacy. The cache is per AirTrail
URL, for 10 minutes. A timeout, a 5xx or a network error fails that
call only. A 401, a 403 or a page that is not the discovery document
uses legacy for that call only. A 404 for one flight makes the client
ask the probe again before it reports the flight as deleted.
A missing scope gives AirtrailScopeError, which names the scope. It
does not unlink a flight. The API key hint and the help step list
the scopes for read and for write-back, in all languages.
Each new request goes through safeFetch, with the SSRF guard, the
timeout and the size cap. The flight list reads pages of 100 and
fails above 50 pages. A failed list unlinks nothing.
airtrail-v1.ts translates between the v1 shapes and the legacy
shapes. A write-back looks up the airport, airline and aircraft ids,
and changes each local time to a UTC instant in the zone of its own
airport. It writes each instant in the +00:00 form of the legacy
save, because AirTrail stores the string as it gets it and the sync
hash reads that string.
The tests cover both APIs over HTTP, the probe cache rules, the sync
hash on a switch and on a save, the page limit, the scope error, and
the UTC conversion across midnight, the date line and daylight-saving
changes.
Adding a day before a trip's start with "Keep bookings on their dates"
moved almost everything a day earlier and emptied the last day. The day
grid was always rebuilt by position: the n-th day row took the n-th date
of the new range. With bookings kept on their dates, only timed bookings
and stays were re-anchored afterwards. Places, day notes, bookings with no
time and a booking's place in a day's order stayed on their re-dated rows
and slid to the day before. Days added at both ends had the same problem.
planDayGrid now takes keepDates. When the new range still holds every
dated day, each of them keeps its date and the new days go around them.
The server passes it unless the traveller chose to shift everything, and
the trip dialog's warning check uses the same rule. A moved or shorter
range still follows the positional rule the dialog describes, and "Shift
everything" still moves the whole plan with the new start.
Two things keyed outside the day rows followed the old rule. Road trip day
boundaries are keyed by day number, so a new first day left each one on
the day before its own. They now follow their days to their new numbers
and go out to every screen. The owner's leave days moved by the change of
start even when days were only added. They now move by as far as the
trip's first day moved, which is nothing when days are only added. The
help text of the option and the MCP description say what happens when
days are added.
Closes#3119
The refusal listed the differences and nothing else. A table or column
only one side has now comes with what that means: the file is at this
version's level, the start that brought it there added every column it
could, the start log names the rest, and the operator keeps running on
SQLite and reports it. A column of TREK's own schema the move cannot copy
is listed apart and named a bug in TREK, since the file is not at fault.
The server guide describes the gap, the repair, the check and the refusal.
After the migrations, both engines compare the declared entities with the
live schema in one catalog statement and log a warning naming each table or
column that is missing, what fails because of it, and that it should be
reported. It never refuses a start, not even when the check itself fails,
and says nothing on a database the migrations built; a Postgres lane suite
holds that on a real server.
The legacy baseline records the positional runner's steps up to
schema_version as executed without running them, so a column that runner
never added stays missing for good. Two things in its history left such
gaps: a step inserted below a version a build had already shipped (step 53,
users.mfa_backup_codes, so turning on MFA failed there) and a column added
to createTables() with no step (users.immich_access_token).
Migration20200101046460 adds, where missing, every column the schema held
at legacy step 258 that head still holds and ALTER TABLE ADD COLUMN can
add, declared as head declares it; on Postgres every one of them is in the
baseline, so nothing is added there. The list is derived again from the
migrations in tests/unit/db/legacy-column-repair.test.ts, which also adds
all of them to a database lacking them and upgrades a v244 install shaped
like the one whose move to Postgres was refused.
Controllers read their path ids through the rowId pipe now, so unit tests that call a handler directly pass the number the pipe hands it. The e2e suites enable addons with true, the type of the entity's field. The realtime gateway test builds its book sockets through one helper instead of a second hand-built TripAccessService.
Comments and test titles added on this branch cited review labels and
planning documents that do not exist in the repository, and some used
dashes as separators. They now say what the code does and why in plain
sentences, and the Postgres lane's suites share one header format. Test
ids stay as they were.
The boot refusal beside a SQLite install told the operator to run the move
with plain `docker compose run`, which uses the SQLite compose file. It now
gives the command the wiki gives, and PGBOOT-037 holds both to the page.
The move read unique indexes by joining indkey to pg_attribute. An expression key such as lower(email) has attnum 0 there, so idx_users_email_lower was missing from the plan. The catalog query now reads each index's key as Postgres prints it, whether any part of it is an expression, and the columns it reads: the plain key columns and the Var nodes of indexprs, without INCLUDE columns or the predicate.
A broken unique key no longer names --drop-nul as its cause. The refusal names the key and the shared value from Postgres's detail. It explains that the SQLite file held the two rows apart, and the hint about SQLite's lower() appears only for an expression key. Values --drop-nul changed in the key's columns are listed as rows to check first, and only when the duplicate value holds their text. An unrelated value that lost a NUL is no longer named. A plain key over a quoted column is named by its column, not as an expression.
A row that breaks a constraint (SQLSTATE class 23) is now a MoveRefusal that says what to correct, in the SQLite file or in a plugin's plugin.db. Before, it was a bare error with a stack. A value Postgres refuses by its type (class 22) stays a failure: it names no row, and values.ts should have refused it first, so the message asks for a bug report. MoveRefusal and the remedy text move to refusal.ts so copy.ts can use them without an import cycle. The usage text of --drop-nul and the wiki's move section describe NUL in JSON, the key listing and these refusals.
A raw U+0000 in TREK's JSON text, which an old or hand-edited travel.db can hold, was refused as "not valid JSON" without naming the NUL or the remedy. It is now refused with the NUL reason, so the refusal names --drop-nul. With --drop-nul, raw and escaped NULs are removed and nothing else, the way TREK's own writes drop them, and each change is listed. Text that is not valid JSON even without its NULs is refused as that, and the checksum's source side follows the same rule.
A \u escape of half a surrogate pair parses in JavaScript but jsonb refuses it. Before, that failed the insert without naming a row. It is now refused by table, row and column.
The plugin SQL guard now refuses what still reached outside a plugin's
schema or depended on the server:
- reads of the system columns (xmin, xmax, cmin, cmax, ctid, tableoid),
which show transaction ids and storage layout and do not exist on
SQLite; a value of the plugin's own may still be named xmin with AS
- the xid and xid8 types wherever the word stands for the type, since
age() of an xid is the cluster's transaction counter
- the text search functions that coerce a text argument to regconfig, a
catalog lookup by name, which leave the allowlist
- collations and access methods outside the set every server has, which
could tell whether a locale or an extension exists
Every plugin transaction pins, with SET LOCAL, the settings that decide
how values print and parse (TimeZone UTC, DateStyle, the C locales, float,
bytea and xml output, the text search configuration), so a plugin's output
neither depends on nor reveals the server's configuration.
Both stores now keep one contract. Its texts and caps live once, in
plugin-data-contract.ts, and both stores throw only those: NUL in SQL,
an argument or a migration id is refused on SQLite as on Postgres,
transaction control is refused in every call on both engines (tx() is
how a batch becomes atomic), a closed store refuses every call, and the
Postgres store answers the cases better-sqlite3 names with its words.
The SQLite store moves beside the Postgres one as SqlitePluginDataStore
in db/engine/sqlite, takes its file from the backend so src/db never
imports src/nest, and no longer needs the driver exemption. A contract
test runs every violation on both stores and fails when they answer
differently. The Postgres lane holds the allowlist, the pinned settings
and the SQL plugins commonly write against a real database.
ROUTEID walks every route the app registers.
- ROUTEID-001 sends 'abc' and the non-canonical spellings of a seeded row's id to each path param named like an id, and requires no 5xx and the answer 'abc' gets. It then sends the spellings of an id no row has and records what the database drivers are handed (tests/helpers/sql-capture.ts, below MikroORM and Kysely). It fails when a spelling, or the number Number() makes of it, reaches SQL, so a route over a table the suite seeds nothing in is held too. Stale NOT_ROW_IDS entries fail, and the walk runs on one listening keep-alive server.
- ROUTEID-002 walks every @trek/shared request, body and query schema and every route's body and query DTO, and fails on a field named like a row id that accepts such a spelling. String ids are listed with their reason; open fields name the handler test that holds them.
Three more suites cover the other entry points:
- MCPROWID does the same for every MCP resource template.
- RPCROWID refuses a plugin RPC id param read past rowIdParam.
- BODYID, JRNID and QUERYID cover the open accommodation update body, the collab reply, the journey fields a handler parses itself and the notifications limit and offset.
All suites that boot the app run in the SQLite suite and in the Postgres lane on 14 and 18. server/CLAUDE.md states the rule and the four gates.
Eleven MCP tool files carried their own parseId, which used Number() and so took '0x1' and '1e20'; they now share uriRowId. The websocket join and book:join read the id with toRowId instead of Number().
Plugin RPC ids go through rowIdParam instead of num(), and a param that may be left out goes through the new optionalRowIdParam. num() let '0x5', 1.5 and 1e20 through, which Postgres refuses or reads its own way while SQLite matches nothing.
accommodations.create read its refs with Math.trunc(Number()), and accommodations.update bound its open input's refs as sent. collab.createMessage took replyTo only when typeof was 'number', which let 1.5 and 1e20 through. All three now read their ids as row ids. A canonical numeric string still passes, as the published contract says, and the refusal keeps num()'s message.
The services and controllers that took an id from a request with Number(), parseInt() or not at all now use toRowId. That covers admin users, notifications (and their limit and offset), trip members and guests, day notes, assignments, memories (photo owner, album link, body photo_id), photos, mail-ingest, the assistant id parsers, dawarich, registration invites, the plugin contribution routes and the collab reply.
The accommodation update body is open, so AccommodationsService reads the place and day refs of every stay write with toRowId and answers a refused one with the create schema's 400.
AdminService.deleteUser checked the caller with parseInt and deleted with Number(), so an admin could delete their own account by spelling its id in hex; both now use one parse.
The OAuth session revoke called Number(id) first and so revoked session 5 for '0x5'; the service now reads the id as sent.
Request fields that hold a row id took z.union([z.number(), z.string()]), a bare z.string() or z.array(z.number()), so '0x5', '5e0', 1.5 and 1e20 reached the handler and from there SQL, where SQLite matches nothing, Postgres 14 and 15 refuse the cast and 16 and later read hex and underscores as a number. Two primitives join idParamSchema: rowIdInputSchema takes a safe integer or its digits and keeps the value as sent, so field types do not change; rowIdStringSchema takes digits only, for multipart fields. Both refuse with 'Invalid id'. The accommodation, assignment, reservation position, admin invite, collab reply, vacay, file and memories photo ids now use them, and every list of ids is z.array(z.number().int()). idParamSchema takes only the decimal spelling or a whole number, and the trip-journal-link and tour library ids use it. A body id spelled '0x5' is now a 400 on SQLite too.
RowIdPipe (src/nest/common/row-id.pipe.ts) parses a path param with toRowId. It answers a refused spelling with the route's own not-found answer, so '0x5', '5e0' and 'abc' get what an id that names no row gets. ParseRowIdPipe replaces Nest's ParseIntPipe with the same 400 and message: ParseIntPipe let '-1' and numbers past 2^53 through, and Postgres refuses the latter as out of range. Journey, journey recordings, day import, book design, map, public photo, collections, doc-sync, vacay, atlas and the plugin user-data export declare their ids through it. The upload routes parse in the handler with withRowId, so a refused id takes the existing cleanup path, or carry SpoolCleanupInterceptor (collections).
toRowId moves to src/db/row-id.ts so repositories can use it (src/db sits below src/nest), together with a new withRowId helper; src/nest/common/row-id.ts re-exports both. Its docstring is rewritten to state the rule plainly. TripsRepository read its number | string trip ids raw: SQLite matched '0x1' against nothing, Postgres 14 and 15 refused the cast with a 500, and 16 and later read it as trip 1. Its access reads, DaysRepository.existsInTrip and the packing template lookups now parse the id first, so an id toRowId refuses is no row on every engine. TripAccessGuard, TripOwnerGuard and JourneyAssistantGuard used Number(), which read '0x1' and '1e0' as row 1; they now use toRowId.
An attack on the move to Postgres with real SQLite files found five values
that changed without a refusal or a listing, or a valid file that failed:
- Negative zero in a plugin column became 0: pg sends the number -0 as "0",
and the checksum printed both as "0". It is sent as text and kept apart in
the checksum on both sides.
- Text whose bytes are not valid UTF-8 was read with each bad sequence
replaced by U+FFFD, and the checksum was built from the replaced text.
Text is now read as its bytes and decoded strictly in the file's encoding
(UTF-8, or UTF-16 for a file created so); such a value is refused in
TREK's and in plugin data, with --drop-nul too, and listed with its bytes.
- --drop-nul serialized a JSON value again, so a large number lost digits
and 1.0 became 1. Only the \u0000 escapes are removed now.
- A plugin default Postgres refuses for the column (DEFAULT 0 on a BLOB
column) failed the move, and DEFAULT 1.5 on an integer column would have
been rounded. Such a default is left off and listed as not carried over.
- A plugin index or primary key SQLite compared with COLLATE NOCASE or RTRIM
became one that compares exactly. It is still created, and the summary
lists the collation it lost.
Each case has a regression test that fails without its fix, on SQLite and
on the Postgres lane.
Postgres reads x.f as the call f(x) when x has no column f. The guard counted
that spelling only after ")." or "].", so over a function in FROM that
returns a single value, or a parameter, the allowlist was bypassed:
"select g.obj_description from generate_series(...) g" returned the comments
on every object of the database, and g.current_schemas, g.txid_status and
$1.ts_token_type passed as well. A refused catalog function after a dot is
now refused too, except the names in COLUMN_NAMES (money, name, oid, version,
xid, xml), which take no argument or only convert it to their type, so t.name
keeps working. PGPLUG-021 holds that list to pg_proc on the lane's server.
The type aclitem names roles: '=r'::aclitem printed the cluster's first
superuser and 'x=r/x'::aclitem told whether a role exists. It is refused in
every spelling like the reg* types.
The allowlist itself moves to assets/postgres/plugin-functions.txt, beside
the catalog list it is checked against. As TypeScript it spelled 'like' and
'json_each', names of Postgres functions that the SQLite dialect gate reads
as SQLite SQL.
The booking body is not schema-checked, so day_id, end_day_id, place_id,
assignment_id, accommodation_id and the create_accommodation ids can arrive
as any string. The reference guards passed them to SQL as they came. SQLite
compares '0x5', '5e0' or 'abc' against an integer id and matches nothing
(or, for '5e0', the row), so the guard answered "Unknown reference". Postgres
refuses the cast on 14 and 15 and reads hex as a number from 16 on, so the
same body answered a 500 on one major and reached a real row on another;
the Postgres 14 lane caught it in L2.
Both guards now read each id through toRowId, the rule every other untrusted
row id follows: an id it refuses resolves to nothing, so referencesOutsideTrip
does not name it and unresolvedReferences does, and the answer is the same
400 on every engine. create() reads the payload's accommodation_id through
toRowId as update() already did, instead of storing '0x5.0', which the
reads' join could not cast back on Postgres.
ReservationRow and ReservationTraveler move to reservation-rows.ts, the way
reservation-inputs.ts holds the input shapes, so the service stays within
its size baseline; the service exports them on and it is lowered to 1166.
The fakes that answer safeFetch by API took their init as any; safeFetch
declares it as RequestInit, which is what they read (the body as
URLSearchParams or text). The SSRF album case had gained one more of them,
which put the tests' no-explicit-any count one over its baseline. All five
are typed now, and the baseline is lowered to the 1545 the tests hold.
The plugin in the move suite gains a BLOB column holding text and an
untyped column holding numbers, and the plugin store must read both back
as they were stored. A new suite moves a file whose plugin and own rows
hold NUL: the plugin's refuses before anything is written, TREK's
refuses until --drop-nul, a username that dropping the NUL makes
duplicate is named with users_username_unique, and a move without a
collision lists the value it changed and stores it without the NUL.
The move stripped NUL from a plugin's own text, merging keys that were
distinct, and turned a plugin's values into another type (text in a BLOB
column into bytea, numbers in an untyped column into text). Its checksum
was built from the converted values, so both passed as verified.
A plugin's data now moves exactly as the Postgres plugin store would
hand it back, or not at all: a NUL, a value of another storage class
than its column, or an integer past 2^53 refuses the move before
anything is written, listing plugin, table, row and column. A column
left open by SQLite's affinity takes the kind its values need, so text
in a BLOB column stays text.
TREK's own text holding a NUL from an older install is refused by
default too. --drop-nul drops it, lists every changed value in the
summary, and names a unique key that dropping a NUL broke.
Every table's checksum is now built from the values read from SQLite,
so a conversion that changes a value fails the verification.
nulFreeInput left a string as sent under any key containing "password".
That covered the admin settings password_login and password_registration,
which are flags: 'true\u0000' failed the lockout check as a disabled
password login while the database stored 'true'. It also tested the raw
key, so 'pass\u0000word' lost the NUL of the password it carried.
The exemption is now HASHED_PASSWORD_KEYS, the fields TREK only ever hashes
with bcrypt (password, current_password, new_password), matched exactly on
the key without its NUL. A unit test reads every field of the shared
schemas and the instance settings named like a password, passphrase or
secret and fails until each is either in that list or decided against with
its reason.
The plugin SQL guard refused functions by name, so every function that reads
the catalogs or the server and was not listed passed: obj_description,
col_description and shobj_description returned the comments on TREK's
tables, columns, database and roles by OID, format_type named a type by its
OID, and version(), current_database(), inet_server_addr() and the session
keywords described the server.
A call to a pg_catalog function is now refused unless the function is on
PLUGIN_FUNCTIONS (string, number, date and time, JSON, array, aggregate,
window, UUID, range, geometry, network and text search functions), in
either spelling of a call, f(...) and the column notation (x).f. The
catalog's function names of Postgres 14 to 18 are kept in
assets/postgres/catalog-functions.txt, and PGPLUG-021 fails on the lane
when the server has a function that list does not name, so a new major
cannot widen what a plugin may call. The session keywords current_user,
session_user, current_role, user, system_user, current_catalog and
current_schema are refused unquoted.
The token guard kept leaking: on a superuser connection an unqualified
`alter type int4 rename to int4x` renamed pg_catalog's type for the whole
server, sequences and catalog-only objects grew the database past the quota,
regtype/regconfig/regdictionary looked TREK's objects up by name, and a
column alias named insert or merge in front of INTO made a table. Instead
of a longer word list, the store now asks the database:
- No plugin SQL while TREK's role is privileged in the sense of the restore
check (PRIVILEGED_ROLE_SQL, imported from server-checks.ts): every db:own
call fails with PRIVILEGED_ROLE_PLUGIN_REFUSAL, read once per pool
connection, and the boot log says why.
- The objects an ALTER, DROP, REFRESH, TRUNCATE or CREATE INDEX ... ON
changes, and the tables a CREATE or ALTER TABLE ties to its own, are
resolved with to_regclass/to_regtype in the transaction right before the
statement, and it is refused unless each one found lies in the plugin's
schema. An unqualified int4, text or box is pg_catalog's and refused even
with the role check off.
- The quota counts every relation of the schema with its TOAST table, and a
schema holds at most PLUGIN_SCHEMA_MAX_OBJECTS catalog objects (relations,
types, constraints, enum labels).
- The guard refuses every reg* type and its array in every spelling, the
has_*_privilege functions, currtid2 and the BRIN/GIN maintenance
functions; INTO only after the INSERT or MERGE that is the statement's
verb or a WITH body's; TEMP only where it makes a temporary object
(CREATE TEMP/TEMPORARY, SELECT ... INTO TEMP), so a column named temp
works again; a base type; and a NUL anywhere in the SQL text.
- A bound value or a migration id holding NUL is refused with the reason
instead of being stripped, so the plugin's keys never change meaning.
Every bypass from the re-review is a test on the real database, on the
role the plugin data is meant to run as and on a superuser connection with
the role check off.
5ff5e7660 stripped U+0000 from every argument a plugin bound to its own
SQLite file, but left the rows it had stored alone. A key written with a
NUL (through a bound arg or char(0) in the plugin's SQL) could no longer be
found by the same key, upserts added duplicates, and keys that differ only
in where a NUL sits ('a\0bc', 'ab\0c') collapsed into one and collided.
A plugin's data is the plugin's, so its file binds the arguments exactly as
sent again, as before that commit. TREK's own data still holds no NUL; the
Postgres store, whose text cannot hold the character, refuses it instead.
The privileged-role check missed three ways a role can reach the database
host from SQL. Before PostgreSQL 16 a member of a CREATEROLE role sets its
role to it and grants itself pg_execute_server_program, since role
attributes are not inherited but the grant checks the role in effect. A role
granted EXECUTE on lo_export, lo_import, pg_read_file, pg_ls_dir and their
siblings reads or writes server files, and lo_export keeps the file after the
transaction rolls back. From PostgreSQL 15 on a role can be granted ALTER
SYSTEM on a setting or SET on a superuser-only one.
PRIVILEGED_ROLE_SQL now counts all three. The parameter grants are read
through query_to_xml inside a CASE taken only on 15 or later, because a
statement naming pg_parameter_acl does not parse on 14. A SET grant on a
setting every role may change anyway, and pg_monitor, stay ordinary. The
lane test creates each kind of role and shows the escalation the check is
for on the server it runs against.