I do something similar. Synching and readera. The books are not that big, so I just keep them all downloaded. I use nextcloud for my audiobook library, but the android app is kinda fiddly. Id.love something simpler that worked like a Google drive so I could browse the files, and only download the ones I want. But of course, hosted on my own computer.