Streaming is Broken V: Hardlinks and Permissions

If you skip this post and head straight to the next chapter, your server would work just fine… for a little while. But then you’d notice the inevitable: You downloaded a 50GB movie, yet your consumed disk space jumped by 100GB.

Whose fault is it? It’s the file system’s inefficiency.

In this post, we’re going to solve our server’s biggest logical bottleneck: data duplication. We will implement the magic of Hardlinks and solve any potential Linux permission issues once and for all.

Hardlinks

In an amateur setup, the flow looks like this:

1. qBittorrent downloads the movie to /downloads.

2. Radarr (which will “acquire” our movies) sees the file, copies it to /movies, renames it to look nice, and delivers it to Jellyfin.

3. Result: You have the dirty file (for seeding torrents) and the clean file (for watching). The same movie takes up double the space.

4. The Cost: Besides eating up half your 512GB SSD, the copy process (I/O) stresses the drive and uses unnecessary CPU.

Who comes to our rescue in this scenario? Hardlinks!

But what the hell is that? Think of a file on the disk not as a “thing,” but as a numerical address pointing to a physical location on the SSD.

• A normal file is a name (movie.mkv) pointing to address 12345.

• A Hardlink is creating a second name (renamed_movie.mkv) that points to that same address.

To the operating system, they look like two different files in different folders. But they occupy the physical space of only one. Cool, right? If you delete one, the other still exists. The space is only freed when the last name is deleted. In this new scenario, what do we have? Instant “copying” (taking a few milliseconds) and zero extra space consumption.

But like everything in life, there is a catch. Hardlinks do not work between different file systems.

In Docker, each mapped volume is treated as an isolated file system.

The Common Error:

• Volume in qBittorrent: /home/user/downloads -> /downloads

• Volume in Radarr: /home/user/media -> /movies

• To Radarr, /downloads and /movies are different disks. It cannot create a Hardlink. It will copy the file. Same problem.

Solving this is simple; we will map the root folder of our server into all containers.

• Host: /home/donkey/EmuleVision

• Container (All): /data

Inside the container, qBittorrent downloads to /data/downloads and Radarr moves it to /data/media. Since both are inside the /data volume, Linux allows the Hardlink. The magic happens.

File Permissions

Linux is heavy-handed with ownership. If qBittorrent (as User A) downloads a file, Radarr (as User B) cannot modify or rename it. Of course, we aren’t going to solve this by running chmod 777 (insecure total permission) on everything.

Instead, we will use the PUID (User ID) and PGID (Group ID) environment variables. In our docker-compose.yml and .env file, we define that all containers run with the identity of your main user (1000).

Nothing new for those who already work with Linux, but it’s always good to remember so you don’t waste time later on.

In the next chapter, we will finally talk about our Docker Compose and all the infrastructure tools we will use on our server.