Aghaster

Bittorrent-based Package Manager

18 posts in this topic

I had an idea the other day while taking my shower and thinking of a future (but distant) OpenSolaris distro I'd eventually like to make.

I had the idea that one could make a package manager for *nix distro that uses the bittorrent protocol to share files between users, thus saving the distro's bandwith. The main servers would host the files and torrents, and uses would share what they have downloaded from it. I also thought that we would have to set a limit of bandwith usage not very high not to take all the connection from a user.

But i also thought on the other hand that this would result in a performance loss for users.

What do you think? Does it already exist?

0

Share this post


Link to post
Share on other sites

IMHO its a good idea, but thats under the idea that users are going to have to manually choose the packages they want, now i dont want 100 packages hanging out in my bittorrent client, now, if there was to say a program that would take care of all of this, and there was one torrent with all the packages in it, where it would choose what it needs then shares it in the background, theres an idea.

but what do i know.

0

Share this post


Link to post
Share on other sites

Users would share only the packages they have downloaded. The most used packages would benefit from a high number of people sharing it (for example, packages for the kernel, gnome or KDE). These packages, as they would be the most popular, are also those that would require the most bandwith. Having lots of people using it would save bandwith. Unpopular packages would still be available from the main server.

0

Share this post


Link to post
Share on other sites

I honestly don't see it being a big hit. It is a great idea, and in an ideal world would work perfectly. I kind of think people would resent the bandwidth usage though, no matter how minimal. Even at 2kbs per installed package, if traffic was constant, it could become a noticable con (to a user). Especially one with a single machine that wants to use VoIP, stream something and maybe even some downloading of their own.

Pros of this are almost exclusive to the would be package distributors server. That guy is the only one who would think of it a purely positive light, and in the world of the selfish, that would probably be the single thing that turns everyone off to it. To bad, it would be nice.

0

Share this post


Link to post
Share on other sites

Yeah... I think you're right. However, this leads me to another idea:

Even if most users would NOT share using this system, the multiple servers could still use the bittorrent protocol. Imagine for example that all debian mirrors served their packages through bittorrent. A user would then download from multiple servers at a time, thus increasing overall download speed. Normal users could choose to enable sharing of their packages on their computer, therefore contributing to the saving of server bandwith (and increase of download speed for other users), but that option would not be forced.

It would be kind of a centralisation of distribution of packages through torrents... and it would increase download speed significantly I think.

0

Share this post


Link to post
Share on other sites

I think you hit the nail on the head there. This would be a way more effective way to do it. I know there would be quite a few people, perhaps hundreds that would both use torrent based package downloads, and maybe they would even run trackers, act as seeders and all that too if the distro was popular enough.

0

Share this post


Link to post
Share on other sites

I'm currently following the Linux From Scratch ebook to make my own Linux system from scratch. From there I could start writting my own package manager, and maybe one day this idea I had while taking my shower will become real. Ideas come out of nowhere sometimes.

However, this day is probably quite distant from now. If anybody is interested in trying to implement that concept, I'd definitly like to know!

0

Share this post


Link to post
Share on other sites

i dont suggest making your own LFS just because you have a good package manager doesnt mean you have a good distro, i mean im running slackware, if i REALLY wanted to i could install portage and ill be fine. try just the package management first then the world.

good luck either way :) im interested btw, what language you thinking of writing this in?

0

Share this post


Link to post
Share on other sites

I had an idea the other day while taking my shower and thinking of a future (but distant) OpenSolaris distro I'd eventually like to make.

I had the idea that one could make a package manager for *nix distro that uses the bittorrent protocol to share files between users, thus saving the distro's bandwith. The main servers would host the files and torrents, and uses would share what they have downloaded from it. I also thought that we would have to set a limit of bandwith usage not very high not to take all the connection from a user.

But i also thought on the other hand that this would result in a performance loss for users.

What do you think? Does it already exist?

I think the reason this isnt implimented in current distro's is for a couple of reasons, in short

#1 Stability, there are distro's who still use lilo and standard partitioning rather than grub and LVM. Change to new ideas and protocols in major *nix distros takes forever.

#2 Control, many distro's even though they embrace the open model which to control the distro as far as #1 and some would even prefer if you would buy there cd. I also belive that control of the licence is paramount ot the major distro's. Liability is a issue with even non comerical distros.

#3 Security, everyone who is peering broadcasts there ip, The packages could possibley (however remote) be tampered with or mangled.

#4 Practicality, bittorrent however popular dosnt offer any incentive nor protocol for folks to properly seed and there is little incentive for a greedy user to continue to seed ,notice how torrents eventualy die off? It is also trackerless which also adds to the difficulty.

That is my view as to why it hasnt been done YET, I do think it could be done , however you would need to alter the bittorrent protocol to address various issues and have a very informed user base.

0

Share this post


Link to post
Share on other sites

Nice idea, personally I don't mind seeding and would use this. But to bad I can't help you code it though

0

Share this post


Link to post
Share on other sites

Nice idea, personally I don't mind seeding and would use this. But to bad I can't help you code it though

Meh, I've been installing Sourcemarge tonight, and while reading about it I read their package manager, written in bash, has 30 000 lines of code. I wonder how big a bittorrent-based package manager would be.

0

Share this post


Link to post
Share on other sites

Meh, I've been installing Sourcemarge tonight, and while reading about it I read their package manager, written in bash, has 30 000 lines of code. I wonder how big a bittorrent-based package manager would be.

Well you wouldn't have to reinvent the wheel, you'd use the standard communcation protocol so you'd just need a decent front end for it. I think the idea has merits but the practicality is somewhat limited from a user perspective. Unless you have unlimited bandwidth it could chew your allowance quickly (I'm talking about unfortunates like myself and 60% of Australia who have both upload and download counted).

Also from a Sysadmin point of view it would be a bit of a headache. How would I differentiate legitimate updates from illegal file sharing? Assuming I redirect my ports to a single host (or pool of hosts) I am still opening up an unecessary security hole into my network. It *could* be managed but the overhead would outweigh the benefits.

Then you have the issue of package reliability. I can have a fair degree of confidence in packages from official Gentoo, FBSD, Ubuntu etc. servers. MD5 summing is automated. But if it is distributed you would still need a centralised md5 repository which knows how to match up with bittorrented packages.

ISO's are already distributed via bittorrent, but I would be very wary of using it for my package management.

0

Share this post


Link to post
Share on other sites

I could see some users viewing this as an annoyance, however, what if an 'on/off' switch was implemented. Something that would turn the bt offline while the computer was active, perhaps only turning it 'on' while the computer was idle. (which could be automated) You could implement settings along with this, to set different speeds at different times, based on cpu load or the like.

Just my $0.02.

-fs

I honestly don't see it being a big hit. It is a great idea, and in an ideal world would work perfectly. I kind of think people would resent the bandwidth usage though, no matter how minimal. Even at 2kbs per installed package, if traffic was constant, it could become a noticable con (to a user). Especially one with a single machine that wants to use VoIP, stream something and maybe even some downloading of their own.

Pros of this are almost exclusive to the would be package distributors server. That guy is the only one who would think of it a purely positive light, and in the world of the selfish, that would probably be the single thing that turns everyone off to it. To bad, it would be nice.

*edit: spelling

Edited by foldingstock
0

Share this post


Link to post
Share on other sites

Yeah... I think you're right. However, this leads me to another idea:

Even if most users would NOT share using this system, the multiple servers could still use the bittorrent protocol. Imagine for example that all debian mirrors served their packages through bittorrent. A user would then download from multiple servers at a time, thus increasing overall download speed. Normal users could choose to enable sharing of their packages on their computer, therefore contributing to the saving of server bandwith (and increase of download speed for other users), but that option would not be forced.

It would be kind of a centralisation of distribution of packages through torrents... and it would increase download speed significantly I think.

I'll drink to that.

0

Share this post


Link to post
Share on other sites

I must say I really do like the idea.

I have always thought that it would be a good idea to extend the bittorrent protocol to general web browsing, so that when a website is under heavy load, the bit torrent protocol kicks in (heavy demand means more users, hence more sharers) and the files are shared. Under low usage, the browser default to http. Bah just a thought.

0

Share this post


Link to post
Share on other sites

From the answers I see, some people would use such a package manager, but the general public probably wouldn't. So my guess is that this idea has some potential that could be used.

0

Share this post


Link to post
Share on other sites

The only problem I see is when I just want a single package to update. It wouldnt be worth opening a dozen connections for a single file, especially considering the size of most linux packages.

It would be nifty to see a service that understood what packages you needed to update then generate a tardball on the fly. At that point a package manager could take over again and download that file over torrent.

Bittorrent is a pretty connection intensive protocol and most consumer routers *could* be crippled.

I know there are many times that cable users can barely voip and torrent at the same time. Now imagine a production network. More bandwidth true, but more connections on the router as well.

0

Share this post


Link to post
Share on other sites

HTTP/FTP are better protocols for packages. However, BitTorrent is ideal for distributed sharing of ISOs and video files.

The big distros have enough mirrors to make the HTTP/FTP thing reliable. Those mirrors have plenty of bandwidth and speed to accomidate everyone. If a package is not present in the repositories, you can always find its home page with Google and compile it manually. You could also create a package and perhaps submit it to the repositories.

Edited by j4mes
0

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now