Hiyori CLI is being developed to create and maintain HiyoriDB — a relational Anime and Manga Database.
Warning
This project is a work-in-progress and is not production ready.
Hiyori CLI has two main primary functions:
- Ingesting transformed data based on the Hiyori Schema ingested from available sources
- Combining the ingested data into HiyoriDB
Hiyori also collects relational mappings from other sources (if available) and uses that to populate references
in the schema.
Supported references have parsers and are able to extract the ID of the entry for that source and store them in reference_ids
.
WIP. 🚧
- Install PHP8.2+, MongoDB, MongoDB PHP Driver, Composer
git clone https://github.com/Hiyori-API/hiyori-cli.git
cd hiyori-cli && composer install
chmod +x hiyori
- Run commands
You can use the cli tool as ./hiyori
, php hiyori
or php src/run.php
.
Note
It's recommended to keep a 1-second delay between requests for sources to prevent rate-limiting.
php hiyori ingest myanimelist --delay 1
php hiyori ingest kitsu --delay 1
php hiyori ingest anilist --delay 1
Note
General spoiler or Media spoiler tags are not ingested.
Note
Run the combiner after the ingestion of multiple sources are complete.
php hiyori combine {base} --strategy {strategy}
- {base} the name of the source to dataset to use
- There are multiple combining strategies available. The default one is
relational_mapping
.
// example
php hiyori combine myanimelist
// specify a stratgegy
php hiyori combine myanimelist --strategy relational_mapping
This strategy simply checks for matching source IDs across available source datasets and combines values based off of those.
This strategy cross-checks all reference_ids
across available source datasets in hopes to find a matching ID between sources.
Warning
This is not yet implemented.
This strategy compares other metadata across available source datasets. The metadata properties are given weights and are cross checked using various algorithms. Based on this, a score is given. If the score is above a certain threshold, it is merged.
Common data with different representations like status (MAL: "Finished Airing", Anilist: "Finished") are transformed into a common value via Hiyori's Enums.
Property | Data Type | Remarks | Nullable |
---|---|---|---|
title |
String | Entry's main title | ❌ |
synonyms |
Array of String | All other titles (combined from sources) | ❌ |
type |
String | Entry type [tv , movie , ova , ona , special , music ] |
✅ |
episodes |
Integer | Number of episodes (if any mentioned) | ✅ |
status |
String | Status of entry [finished , airing , to_be_aired , cancelled ] |
✅ |
season |
Nullable String | Release Season [winter , summer , spring , fall ] |
✅ |
year |
Nullable Integer | Release Year | ✅ |
images |
Array of String | Default Image URLs (combined from sources) | ❌ |
reference_ids |
Object | Parsed IDs from Supported References | ❌ |
references |
Array of String | Reference URLs (combined from sources) | ❌ |
tags |
Array of String | Genres/Tags (combined from sources) | ❌ |
TBD.
The following URL type, if detected in an entry's references
, will be parsed.
Reference | Shorthand (as returned) |
---|---|
Amazon Prime | ap |
AniDB | adb |
AniList | al |
AnimeNewsNetwork | ann |
Bangumi | b |
Crunchyroll | cr |
Funimation | fm |
Kitsu | k |
MyAnimeList | mal |
Netflix | nf |
Syoboi | sb |
Trakt | t |
TubiTV | ttv |
TVTokyo | tvt |
tx |
|
Wikipedia English | wpen |
Wikipedia Japanese | wpjp |
YouTube | yt |
Right now the main focus is to integrate the initial 3 sources and build a combined relational Anime metadata DB.
- Tests
- Sweepers - Remove dead entries (or create a separate collection of them)
- Ingestion: Allow re-try of failed requests
- Ingestion: Alow resume
- Caching
- Ingestion: Allow flag for metadata update
- EV for secrets
- Logging
- Tool: Combiner
- Tool: Export
- Manga Relational DB
- Dependency Injection / Service Containers
- MyAnimeList Integration via REST API
- Kitsu Integration via REST API
- AniList Integration via REST API
Why does Hiyori Schema consist of metadata?
In the event an entry from a source does not contain relational IDs or mapping provided by sources, the metadata will be used to find the relation instead.
Important
Hiyori is not affiliated with any sources. You are responsible for the usage of this tool. Please be respectful towards the terms and conditions set by these sources.