10 replies to this topic

[Irongeek]

Anyone know any system tweaks to speed up slow USB Thumbdrives? I've noticed that it's faster to copy one 300MB file than three-hundred 1mb files, is this because of extra edits that have to be made to the file allocation table?

[operat0r]

more read writes are done I have the same prob on large websites with PFF it sucks to wait like 10sec every time you click a window.

You guy know if you can change the temp file path so it runs faster ?

[crackedatom]

Anyone know any system tweaks to speed up slow USB Thumbdrives? I've noticed that it's faster to copy one 300MB file than three-hundred 1mb files, is this because of extra edits that have to be made to the file allocation table?

Yeah...isn't it basically the same as with a harddrive?..You try deleting 300 1mb files compared to 1 300mb file and the 300 small files will take a lot longer.
If you were to zip all those files to make 1 large file (with no compression for the sake of keeping them the same size) it will be the same as the 300mb file.
So, I suppose you could keep all your files zipped together in groups and just extract them as you need them.

[xof7]

I would assume that when you are copying just one file over, it keeps the xfer going for the entire period where if you are doing multiple files, you have to stop after each file, locate, make sure theirs enough space left, then begin copying.

[nullkraft]

From the command line in Windows you can use xcopy instead of copy. It buffers multiple file copies before sending the files. In Linux the only thing I could think of trying is rsync except that it reads the destination files before copying. Also, when mounting flash drives in Linux you can use the 'noatime' option. From the man page:

noatime
					 Do  not  update  inode access times on this file system (e.g, for faster
					 access on the news spool to speed up news servers).

A quick test on 41 files at 300.9 MB took 40 seconds. When mounted with noatime it saved 6 seconds. Not a huge speed up, I guess. I unmounted/remounted between tests to flush any buffers.

[anubis26]

...isn't it basically the same as with a harddrive?

Kinda. Deleting files on a hard drive doesn't actually delete the files, just erases their entry in the table. Since the files are still there, recovery software can 'undelete'. That's why things like shred exist. So it doesn't really matter how large the file is.

[Irongeek]

A few more findings:

1. I thought that my Symantec AV may be the problem, but after removal it made no difference.
2. Read and writes are much faster under Linux than under Vista/XP.
3. It's faster in Vista to mount a Trucrypt volume off the thumbdrive, then run your apps off of it then it is to run the app straight off of the thumbdrive.
4. Changing the device options in control panel from the default "Optimize for removal" to "Optimize for performance".

[jabzor]

4. Changing the device options in control panel from the default "Optimize for removal" to "Optimize for performance".

When you use this option make sure you 'safely remove' the disk or you can easily end up with write errors.

[Irongeek]

Ok, even more interesting test results. I used a tool from http://oette.info/
for benchmarking different files systems on a 512MB Thumbdrive. I set it to write five 1 MB files. Here are the results:


Fat16 16kb allocation unit size
Result:
Write time: 3.4228 Sec.
0.292158466752366 MB/s
Read time: 0.0862 Sec.
11.6009280742459 MB/s


Fat16 64kb allocation unit size
Result:
Write time: 3.3486 Sec.
0.298632264229827 MB/s
Read time: 0.0842 Sec.
11.8764845605701 MB/s


Fat16 default allocation unit size
Result:
Write time: 3.3108 Sec.
0.302041802585478 MB/s
Read time: 0.0862 Sec.
11.6009280742459 MB/s
Result:
Write time: 3.3208 Sec.
0.301132257287401 MB/s
Read time: 0.0762 Sec.
13.1233595800525 MB/s


Fat32 default allocation unit size
Result:
Write time: 23.81 Sec.
4.19991600167997E-02 MB/s
Read time: 0.08 Sec.
12.5 MB/s
Result:
Write time: 23.7444 Sec.
4.21151934771988E-02 MB/s
Read time: 0.1202 Sec.
8.31946755407654 MB/s

NTFS default allocation unit size
Result:
Write time: 0.8712 Sec.
1.14784205693297 MB/s
Read time: 0.0682 Sec.
14.6627565982405 MB/s
Result:
Write time: 0.9174 Sec.
1.09003706126008 MB/s
Read time: 0.0882 Sec.
11.3378684807256 MB/s



To cut to the chase, NTFS seems to be the fastest, FAT16 (just called FAT in the dialog) next and FAT32 was dog ass slow. I used the HP UFD format tool to do the formating. Any good reason not to use NTFS on the thumbdrive besides compatibility with other OSes?

[Irongeek]

I did some more test, this time with my 1GB Apacer UFD. I copied
Torpark 1.5.0.7 (27 MB in 275 files, 83 directories) to the UFD as a more organic test:

Fat32
1:05

fat16
0:55

NTFS
1:35
1:23

These new, longer results gave me pause, thats why I ran it twice. I also test NTFS under the following conditions:

No AV live protection
1:11

AV on, using XCOPY
1:38

Compression on
1:17

Seems like turning compression on (less total data to write to the disk) and AV off seems to help. Over all, seems like FAT 16 is the best choice for faster thumbdrive file systems.

Edit:
After testing some of my other drives:
1GB XP FAT16
Result:
Write time: 11.0596 Sec.
9.04191833339361E-02 MB/s
Read time: 0.1406 Sec.
7.11237553342816 MB/s
Result:
Write time: 10.5154 Sec.
0.095098617266105 MB/s
Read time: 0.1376 Sec.
7.26744186046512 MB/s

2GB FAT16
Result:
Write time: 3.528 Sec.
0.283446712018141 MB/s
Read time: 0.1282 Sec.
7.8003120124805 MB/s

4GB FAT16
Result:
Write time: 27.4376 Sec.
3.64463364142636E-02 MB/s
Read time: 0.1 Sec.
10 MB/s
Result:
Write time: 23.7406 Sec.
4.21219345762112E-02 MB/s
Read time: 0.1094 Sec.
9.14076782449726 MB/s

1GB XP FAT32
Result:
Write time: 11.4752 Sec.
8.71444506413832E-02 MB/s
Read time: 0.1124 Sec.
8.89679715302491 MB/s
Result:
Write time: 11.6156 Sec.
8.60911188401804E-02 MB/s
Read time: 0.122 Sec.
8.19672131147541 MB/s

1GB XP NTFS
Result:
Write time: 5.4874 Sec.
0.182235667164777 MB/s
Read time: 0.122 Sec.
8.19672131147541 MB/s
Result:
Write time: 5.6312 Sec.
0.177582042903822 MB/s
Read time: 0.1156 Sec.
8.65051903114187 MB/s


I did the Tor test again and with the 1GB and it took 1:24, so something must be wrong with my synthetic tests.

Edited by Irongeek, 19 February 2007 - 01:41 PM.

[Irongeek]

Looks like these guys did some tests, and over all FAT16 is the winner:

http://www.anandtech...cle.aspx?i=2549