Rework middlewared pipes so that file descriptors won't leak
File descriptors are allocated here but there is no guarantee that they will be closed eventually. Job is responsible for closing them and it may fail.
fix(middleware): Rework pipes so that file descriptors won't leak:
- Explicitly define what side of pipe job will use (reading or writing). Use two pipes if both is required.
- Make caller responsible for opening and closing pipes. Callee checks if pipes were actually open.
- Add `middleware.run_in_io_thread` for blocking I/O on pipes. Thread pool is unlimited, threads are self-destroyed after being used.
This may be not the best solution in terms of system resources but it leads to much simplier code than using e.g.
https://github.com/mosquito/aio-pipe and doing asynchronous block I/O in python may be even more CPU-consuming.
- Cancel jobs started by `core.download` after 60 seconds of inactivity. It is the only caller that leaves opened pipe after finishing.
- Use shutil.copyfileobj whenever possible.
- Fix a few rare-occuring bugs