Skip to content

New upstream fields #906

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
BrutuZ wants to merge 2 commits into
base: main
Choose a base branch
from
Open

New upstream fields #906

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
1b6ae95
Add New Fields
BrutuZ Jan 24, 2026
0677ea7
fix migration, improve code structure in update.ts migration runner
beeequeue Feb 19, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
75 changes: 74 additions & 1 deletion docs/openapi.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -115,6 +115,13 @@ $defs:
minimum: 0
maximum: 50000000
example: 1337
animenewsnetwork:
Copy link
Owner

@beeequeue beeequeue Feb 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please make sure that the order of fields in relations (both in docs and code) are always alphabetical

Copy link
Author

@BrutuZ BrutuZ Feb 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe MAL was at the bottom in one place and it threw off my perception of anything else being alphabetized, I only noticed it yesterday with fresh eyes while splitting the PRs

oneOf:
- type: "null"
- type: integer
minimum: 0
maximum: 50000000
example: 1337
notify-moe:
oneOf:
- type: "null"
Expand All @@ -129,20 +136,55 @@ $defs:
minimum: 0
maximum: 50000000
example: 1337
themoviedb-season:
oneOf:
- type: "null"
- type: integer
minimum: 0
maximum: 50000000
example: 1
thetvdb:
oneOf:
- type: "null"
- type: integer
minimum: 0
maximum: 50000000
example: 1337
thetvdb-season:
oneOf:
- type: "null"
- type: integer
minimum: 0
maximum: 50000000
example: 1
myanimelist:
oneOf:
- type: "null"
- type: integer
minimum: 0
maximum: 50000000
example: 1337
simkl:
oneOf:
- type: "null"
- type: integer
minimum: 0
maximum: 50000000
example: 1337
animecountdown:
oneOf:
- type: "null"
- type: integer
minimum: 0
maximum: 50000000
example: 1337
media:
oneOf:
- type: "null"
- type: string
minLength: 0
maxLength: 10
example: TV

nullable_relation:
oneOf:
Expand All @@ -153,15 +195,20 @@ $defs:
example:
anidb: 1337
anilist: 1337
anime-planet: spriggan
anime-planet: dororon-enma-kun
anisearch: null
imdb: tt0164917
kitsu: null
livechart: null
notify-moe: "-cQb5Fmmg"
themoviedb: null
themoviedb-season: 1
thetvdb: null
thetvdb-season: 1
myanimelist: null
animecountdown: null
animenewsnetwork: null
media: TV
oneOf:
- $ref: "#/$defs/nullable_relation"
- type: array
Expand Down Expand Up @@ -305,6 +352,8 @@ paths:
- anilist
- anidb
- anime-planet
- animecountdown
- animenewsnetwork
- anisearch
- kitsu
- livechart
Expand Down Expand Up @@ -410,6 +459,18 @@ paths:
- type: integer
minimum: 0
maximum: 50000000
animenewsnetwork:
oneOf:
- type: "null"
- type: integer
minimum: 0
maximum: 50000000
animecountdown:
oneOf:
- type: "null"
- type: integer
minimum: 0
maximum: 50000000
- type: array
minItems: 1
maxItems: 100
Expand Down Expand Up @@ -466,6 +527,18 @@ paths:
- type: integer
minimum: 0
maximum: 50000000
animenewsnetwork:
oneOf:
- type: "null"
- type: integer
minimum: 0
maximum: 50000000
animecountdown:
oneOf:
- type: "null"
- type: integer
minimum: 0
maximum: 50000000

responses:
"200":
Expand Down
28 changes: 23 additions & 5 deletions src/db/db.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { mkdirSync } from "node:fs"
import { existsSync } from "node:fs"

import { createDatabase } from "db0"
import sqlite from "db0/connectors/node-sqlite"
Expand All @@ -15,12 +15,27 @@ export const Source = {
IMDB: "imdb",
Kitsu: "kitsu",
LiveChart: "livechart",
AnimeNewsNetwork: "animenewsnetwork",
NotifyMoe: "notify-moe",
TheMovieDB: "themoviedb",
TheMovieDBSeason: "themoviedb-season",
TheTVDB: "thetvdb",
TheTVDBSeason: "thetvdb-season",
MAL: "myanimelist",
Simkl: "simkl",
AnimeCountdown: "animecountdown",
MediaType: "media",
} as const
export type SourceValue = (typeof Source)[keyof typeof Source]
export const NonUniqueFields = [
Source.IMDB,
Source.TheMovieDB,
Source.TheMovieDBSeason,
Source.TheTVDB,
Source.TheTVDBSeason,
Source.Simkl,
Copy link
Owner

@beeequeue beeequeue Feb 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is simkl unique or not?

Copy link
Author

@BrutuZ BrutuZ Feb 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know for sure, no experience with the service but it does the whole "Global Movies and TV oh but also Anime" thing, so I preferred to err on the side of caution with non-unique

Source.MediaType,
]
Copy link
Owner

@beeequeue beeequeue Feb 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this should either be a Map to start with as i said in another comment, or have as const after the closing bracket to improve DevX and prevent modifying it :)


export type Relation = {
[Source.AniDB]?: number
Expand All @@ -30,10 +45,16 @@ export type Relation = {
[Source.IMDB]?: `tt${string}`
[Source.Kitsu]?: number
[Source.LiveChart]?: number
[Source.AnimeNewsNetwork]?: number
[Source.NotifyMoe]?: string
[Source.TheMovieDB]?: number
[Source.TheTVDB]?: number
[Source.MAL]?: number
[Source.TheTVDBSeason]?: number
[Source.TheMovieDBSeason]?: number
[Source.Simkl]?: number
[Source.AnimeCountdown]?: number
[Source.MediaType]?: string
}

export type OldRelation = Pick<Relation, "anidb" | "anilist" | "myanimelist" | "kitsu">
Expand All @@ -43,9 +64,6 @@ export interface Database {
relations: Relation
}

// Ensure SQLite directory exists
mkdirSync("./dir", { recursive: true })

const sqliteDb = sqlite(
process.env.VITEST_POOL_ID == null
? { path: `./db/${process.env.NODE_ENV ?? "development"}.sqlite3` }
Expand All @@ -60,6 +78,6 @@ export const db = new Kysely<Database>({
export const migrator = new Migrator({
db,
provider: new ActuallyWorkingMigrationProvider(
process.env.NODE_ENV !== "test" ? "dist/migrations" : "src/migrations",
existsSync("src/migrations") ? "src/migrations" : "dist/migrations",
),
})
4 changes: 2 additions & 2 deletions src/index.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import { serve } from "h3"

import { createApp } from "./app.ts"
import { config } from "./config.ts"
import { config, Environment } from "./config.ts"
import { migrator } from "./db/db.ts"
import { updateRelations } from "./update.ts"

Expand All @@ -11,7 +11,7 @@ const { NODE_ENV, PORT } = config

const runUpdateScript = async () => updateRelations()

if (NODE_ENV === "production") {
if (NODE_ENV === Environment.Prod) {
Copy link
Owner

@beeequeue beeequeue Feb 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

im not sure about this, it might be better to just remove the Environment constant altogether in another pr/commit

void runUpdateScript()

// eslint-disable-next-line ts/no-misused-promises
Expand Down
2 changes: 1 addition & 1 deletion src/migrations/20190611171759_initial.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import { type Kysely, sql } from "kysely"

export async function up(db: Kysely<any>): Promise<void> {
export async function up(db: Kysely<unknown>): Promise<void> {
await sql`PRAGMA journal_mode=WAL`.execute(db)

await db.schema
Expand Down
12 changes: 12 additions & 0 deletions src/migrations/20260124120530_schema.ts
View file
Open in desktop
Copy link
Owner

@beeequeue beeequeue Feb 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

while i dont know exactly why you had issues with modifying the table, i do vaguely remember that sqlite's ALTER TABLE has a bunch of limitations

ill try to do this myself since im curious

Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
import type { Kysely } from "kysely"

export async function up(db: Kysely<unknown>): Promise<void> {
await db.schema.alterTable("relations").addColumn("themoviedb-season", "integer").execute()
await db.schema.alterTable("relations").addColumn("thetvdb-season", "integer").execute()
// unique, but sqlite can't add unique columns to tables
await db.schema.alterTable("relations").addColumn("animenewsnetwork", "integer").execute()
// unique, but sqlite can't add unique columns to tables
await db.schema.alterTable("relations").addColumn("animecountdown", "integer").execute()
await db.schema.alterTable("relations").addColumn("simkl", "integer").execute()
await db.schema.alterTable("relations").addColumn("media", "text").execute()
}
18 changes: 18 additions & 0 deletions src/routes/v2/ids/handler.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,10 +16,16 @@ const createRelations = async <N extends number>(
imdb: `tt${id++}`,
kitsu: id++,
livechart: id++,
animenewsnetwork: id++,
"notify-moe": `${id++}`,
themoviedb: id++,
"themoviedb-season": id++,
Copy link
Owner

@beeequeue beeequeue Feb 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the seasons should probably be some number between 1-3 rather than an infinitely increasing number, you can hard code it for the tests

thetvdb: id++,
"thetvdb-season": id++,
myanimelist: id++,
simkl: id++,
animecountdown: id++,
media: `${id++}`,
Copy link
Owner

@beeequeue beeequeue Feb 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

media cant be a string number

Copy link
Author

@BrutuZ BrutuZ Feb 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought it was weird, but I followed notify.moe's model that is also alphanumerical and since the tests passed I didn't think about it

}))

// Insert each relation
Expand Down Expand Up @@ -80,10 +86,16 @@ describe("query params", () => {
imdb: null!,
kitsu: null!,
livechart: null!,
animenewsnetwork: null!,
"notify-moe": null!,
themoviedb: null!,
"themoviedb-season": null!,
thetvdb: null!,
"thetvdb-season": null!,
myanimelist: null!,
simkl: null!,
animecountdown: null!,
media: null!,
}
await db.insertInto("relations").values(relation).execute()

Expand Down Expand Up @@ -165,10 +177,16 @@ describe("json body", () => {
imdb: null!,
kitsu: null!,
livechart: null!,
animenewsnetwork: null!,
"notify-moe": null!,
themoviedb: null!,
"themoviedb-season": null!,
thetvdb: null!,
"thetvdb-season": null!,
myanimelist: null!,
simkl: null!,
animecountdown: null!,
media: null!,
}
await db.insertInto("relations").values(relation).execute()

Expand Down
2 changes: 2 additions & 0 deletions src/routes/v2/ids/schemas/common.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@ import * as v from "valibot"
export const numberIdSourceSchema = v.picklist([
"anilist",
"anidb",
"animecountdown",
"animenewsnetwork",
"anisearch",
"kitsu",
"livechart",
Expand Down
3 changes: 3 additions & 0 deletions src/routes/v2/ids/schemas/json-body.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,10 @@ const okCases = [
anisearch: 1337,
kitsu: 1337,
livechart: 1337,
animenewsnetwork: 1337,
"notify-moe": "1337",
myanimelist: 1337,
animecountdown: 1337,
},
true,
],
Expand All @@ -47,6 +49,7 @@ const badCases = [
[{ imdb: 1337 }, false],
[{ themoviedb: 1337 }, false],
[{ thetvdb: 1337 }, false],
[{ simkl: 1337 }, false],
] satisfies Cases

const mapToSingularArrayInput = (cases: Cases): Cases =>
Expand Down
2 changes: 2 additions & 0 deletions src/routes/v2/ids/schemas/json-body.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,8 @@ export const singularItemInputSchema = v.pipe(
anidb: numberIdSchema,
anilist: numberIdSchema,
"anime-planet": stringIdSchema,
animecountdown: numberIdSchema,
animenewsnetwork: numberIdSchema,
anisearch: numberIdSchema,
kitsu: numberIdSchema,
livechart: numberIdSchema,
Expand Down
6 changes: 6 additions & 0 deletions src/routes/v2/include.test-utils.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -74,10 +74,16 @@ export const testIncludeQueryParam = (
imdb: null,
kitsu: null,
livechart: null,
animenewsnetwork: null,
"notify-moe": null,
themoviedb: null,
"themoviedb-season": null,
thetvdb: null,
"thetvdb-season": null,
myanimelist: null,
simkl: null,
animecountdown: null,
media: null,
}
expectedResult[source] = prefixify(source, 1337) as never

Expand Down
2 changes: 1 addition & 1 deletion src/routes/v2/include.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ export const includeSchema = v.object({
v.string(),
v.regex(/^[\-a-z,]+$/, "Invalid `include` query"),
v.minLength(1),
v.maxLength(100),
Copy link
Owner

@beeequeue beeequeue Feb 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is this change here?

Copy link
Author

@BrutuZ BrutuZ Feb 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With the added fields, the query length surpassed this limit.
IIRC it was a test with the include parameter and every field available

v.maxLength(200),
),
),
})
Expand Down
Loading