Receiving a large file over the Internet

Today someone wanted to send me a large video file (around 3 GB). Since we live about 1500 kilometers apart, but both have broadband access, the Internet seemed to be the fastest solution.

Attempt via a file hosting services

Apparently, Google Drive didn't accept such a large file to be uploaded.

The same issue appeared on my Nextcloud instance (hosted on some shared hosting, so I wasn't very surprised).

Attempt via WebRTC/Peer-to-peer in the browser

Trying online services like file.pizza took a while to start and never transferred anything.

Attempt via a Tus server hosted on my computer

What is Tus

At work, I previously set up a solution using Tus.io to allow customers to transfer large files and resume interrupted transfers. This solution has been very reliable, even under bad network conditions.

tus.io - resumable file uploads

So I figured out, that I should set up the same solution, even if it felt like using a sledgehammer to crack a nut.

Playing Lego and assembling the bricks

I first made an upload server using tusd and then I added some frontend using Uppy, which is made by the same company behind tus.

Reference server implementation in Go of tus

Uppy - modular open-source JavaScript file uploader

Transloadit - the company behind tus and Uppy

And soon enough I had an http server running on my machine on port 54321, with an upload page and a tus server.

Transfer in progress

I then made a hole in my firewall and added a forwarding rule to my home router.

I finally sent the link http://[MY-IP]:54321 to the other person, who successfully uploaded the file in a couple of minutes!

One caveat is that the name of the uploaded file is random; you can find the original name in the accompanying .info file.

Conclusion

Assembling the parts took me about 40 minutes, much less than all the trial and error we did before.

On one side, it feels a bit weird that such a basic problem is so hard to solve. However, on the other side, establishing a direct TCP/IP connection between two home routers feels like the right solution to a transfer problem (relying on some third party for intermediate storage is actually absurd).

If anyone wants to solve the same problem, here are my 150 lines of code (most of them are copied from the documentation of tus and uppy).

https://git.sr.ht/~oliverpool/tuspload-me

📅 2021-09-06

Back to the index

Send me a comment or feedback