- Python 96.9%
- JavaScript 2.4%
- HTML 0.6%
| .github/workflows | ||
| docs | ||
| hydownloader | ||
| .gitignore | ||
| .pylintrc | ||
| CHANGELOG.md | ||
| Dockerfile | ||
| LICENSE.txt | ||
| poetry.lock | ||
| pyproject.toml | ||
| README.md | ||
hydownloader
Alternative download system for Hydrus.
What is this and how does it work?
This is a set of utilites that provide a Hydrus-like download system outside of Hydrus, based on gallery-dl. The main component is a daemon that runs in the background and uses a database of subscriptions and URL data to control gallery-dl. It downloads files into a separate folder, independently of Hydrus. It also saves detailed metadata, checks for download errors, keeps track of already downloaded files and provides other features expected from a complete downloader system.
There are separate tools to export known URLs from Hydrus to avoid re-downloading files you already have in your Hydrus database and also for importing downloaded files into Hydrus with metadata. The identifiers of already seen files are stored in a database and are known as "anchors" in hydownloader.
The daemon provides a JSON-based API, which is the primary means of controlling it and querying information from it.
WARNING: Making a downloader system that works with many sites, does not lose/skip files or metadata and correctly handles error conditions is a complex problem. Accordingly, hydownloader is also a complex application that requires investing time and effort to learn. If you are not willing to read documentation and learn, you are better off finding some simpler, less powerful solution.
Graphical User Interface
hydownloader is a server/command line application at its core. It provides a HTTP API, but this is not meant to be directly used by humans, since that would be very inconvenient. Instead, there is a separate graphical user interface that can connect to the hydownloader daemon (either running on the same machine, or even remotely). There are 2 ways to use this graphical interface: there is a web version that is included with the hydownloader daemon, and there is also a standalone application called hydownloader-systray (in fact these two are the same application, once compiled to a native executable and once to WebAssembly to run in the browser). hydownloader-systray has its own separate repository here. The native version is the recommended way to manage hydownloader. The web version is somewhat experimental and might have trouble handling large amounts of data (like big log files).
Documentation - Table of Contents
WARNING: You MUST read the chapters marked in bold before asking for support. If you don't, you will be missing critical information to correctly operate hydownloader and can only blame yourself if things are not working!
- Basic overview of hydownloader and comparison with Hydrus
- Installation, configuration and updating
- Using hydownloader
- Site-specific issues and limitations
- FAQ
- API documentation
- Contributing to hydownloader (incl. site support)
- CHANGELOG
Site support
| Site | Anchor exporter | Subscriptions, single URL downloads | Test |
|---|---|---|---|
| pixiv | yes | yes | yes |
| gelbooru | yes | yes | yes |
| danbooru | yes | yes | yes |
| atfbooru | yes | yes | no |
| aibooru | yes | yes | yes |
| lolibooru.moe | yes | yes | yes |
| 3dbooru | yes | yes | yes |
| artstation | yes | yes | yes |
| sankaku | yes | yes | yes |
| idolcomplex | yes | yes | yes |
| yes | yes | yes | |
| deviantart | yes | yes | yes |
| patreon | no | yes | yes |
| nijie | yes | yes | yes |
| tumblr | yes | yes | no |
| fantia | no | yes | no |
| fanbox | no | yes | no |
| webtoons | no | yes | yes |
| kemono.party | no | yes | no |
| coomer.party | no | yes | no |
| baraag | no | yes | yes |
| pawoo | no | yes | yes |
| hentaifoundry | yes | yes | yes |
| yande.re | yes | yes | yes |
| rule34.xxx | yes | yes | yes |
| e621.net | yes | yes | yes |
| furaffinity.net | yes | yes | yes |
| instagram.com | no | yes | yes |
| redgifs.com | yes | yes | yes |
| mastodon instances | no | yes | no |
| misskey instances | no | yes | no |
| bluesky | no | yes | yes |
| tiktok | yes | yes | yes |
| yes | yes | yes | |
| newgrounds | no | yes | no |
| any other site that gallery-dl supports | no | mostly* | no |
*Downloading works, automatically getting subscription info from URLs and checking file download status through API from the anchor database does not.
Privacy notice
hydownloader contains absolutely no telemetry, analytics, remote logging, usage statistics, install/usage ping or any other kind of spyware. All network usage is according to the user configuration (subscriptions and URLs queued for download). All logs, downloaded data and configuration is stored in the database folder. hydownloader does not read or write outside its database folder, unless requested explicitly by some user-initiated command.
License
hydownloader is licensed under the Affero General Public License v3+ (AGPLv3+).