Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: too high file I/O activity - weak performance - in foobar2k (Read 7256 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

too high file I/O activity - weak performance - in foobar2k

hello there,

so here's the thing: i've noticed an extremely high file I/O activity in foobar2k. this was seen using process monitor (http://technet.microsoft.com/en-us/sysinte...s/bb896645.aspx) capable of logging everything a process does regarding file/registry OS calls.

more precisely, when listening to an mp3, foobar2k performs file reads of 16kb which is not so bad (although still generates a couple of disk accesses / sec), but when flac comes into the equation, things go bad: foobar2k reads chunks of 8k and they come in pair with QueryStandardInformationFile commands. take into account that flac has way higher bitrate and this generates about 20 reads/s and 20 QueryStandardInformationFile/s.

yes, on a regular drive, given that it is sequential access you probably won't feel a thing, maybe the drive/system even performs some heuristic read-ahead optimizations, but when mounting remote drives and playing content from them things start to go bad.
in my case, i have a 'remote' drive on a server located maybe 40-60ms ping away. because of the high number of lengthy reads, it often happens that foobar2k can't read the data it needs in time and playback drops, the file read bitrate gets lower than required playback bitrate at that time. the drops are quite short but they are often.

if copying using total commander for example that file from remote to local, i can see also using process monitor that total commander is reading in 1mb chunks, thus, i get an about 2mb/s bitrate, so let's say around 15mbps which would be enough also for HD video. so the problem is not with the bandwidth. it is clearly an issue of high number of reads/s.

what can be done? are there any settings that can be somehow tweaked or this must be fixed in foobar2k code directly?
the solution is obvious, read much larger chunks in memory when accessing disk, problem solved.

i think EpicForever's problem here is somewhat related: http://www.hydrogenaudio.org/forums/index....howtopic=102374 -- if foobar2k wouldn't abuse disk i/o in some cases and would use larger r/w chunks, i/o performance would be much much better and slow/problematic hdd's + remote drives would also work fine.

can i somehow get in touch with foobar2k developers to report this issue?

thanks much.

edit:
also, if foobar2000 starts rescanning the remote drive placed in media library, the flac playback is obviously a complete disaster.

also attached a screenshot of what those calls look like in process explorer where can we see those 8k chunks and repetitive file queries (paired)

too high file I/O activity - weak performance - in foobar2k

Reply #1 –
Wow... And I have ordered a new mobo to solve my problems

too high file I/O activity - weak performance - in foobar2k

Reply #2 –
well, it may solve your problems because normally you wouldn't see those kind on a perfectly healthy computer. timeouts of seconds generated by your hdd (if i've read your thread well) are not normal, BUT, considering you are using a wd green drive, it may happen.. greens are economical, big storage, cheap price, but their performance is weak. their firmware is tweaked in such a way that operation lengths won't matter much and internal maintenance operations may take a very long time. search for raid and wd greens (but also other slower and non-enterprise drives) and you'll see various problems like disk being thrown out of array or marked as failing because operation timeouts. on enterprise drives, firmware is programmed in such a way that any operation responds in a relatively fast time making them suitable for raid arrays. home use drives like greens take their time to do maintenance and i guess this can generate in some conditions 'hangs' of seconds.
still, if they're often maybe you have other problems...

anyway, i was saying just that foobar2000 way's of frequently accessing the disk in some cases generates my problems for sure and it may also overload your system especially if it doesn't handle many i/o operations/s very well. although tweaking or modifying foobar2000 disk access algorithms may 'virtually' also solve your problem it doesn't mean that your system doesn't have a harddrive/controller/driver problem that manifests under i/o load. from which i've seen i guess nobody complains about problems like ours and everybody accessing music from local drive runs well. i also don't experience problems running from local drive, mine is with remote ones with high response times (because of network).

changing mainboard/drive/drivers SHOULD fix your problem actually in all i/o load cases, but fixing the issue that i'm reporting here will also increase foobar2k's performance from file i/o point of view.

i don't remember exactly if you've tried but make sure that you have a clean install of an untouched OS and install LATEST drivers from your motherboard manufacturer support website.

too high file I/O activity - weak performance - in foobar2k

Reply #3 –
Quote
are there any settings that can be somehow tweaked


have you not looked in the preferences?

file>preferences>network>buffer size
file>preferences>playback>output>buffer length

i don't know if they will help. the defaults have always been fine for me.

too high file I/O activity - weak performance - in foobar2k

Reply #4 –
nope, unfortunately they don't have any effect in this case. flac is still read from the beginning of the file, sequentially, reading 8188 or 8196 bytes at a time.
i've increased network buffer to about 5mb, playback output buffer to about 10s, saved, restarted foobar.
furthermore, increasing those buffers have absolutely no effect at all in foobar's behaviour. it is understandable that network buffer may not have any effect when playing files from network mapped drives because this is a windows OS mechanism of abstracting a remote file as a local one so foobar opens files as 'local' in this case, but i wonder why playback output buffer has no effect. i was expecting something like a delayed play start while loading the first 10s of music and at least 10s of clear uninterrupted playing but this doesn't happen.

anyway, those settings won't affect the way foobar2k reads files in terms of buffer sizes.

too high file I/O activity - weak performance - in foobar2k

Reply #5 –
You don't say exactly how you're connected to the remote drive, which O/S you're using, or which O/S the remote system is using. This is quite important, because what you are seeing looks like it might be related to the CIFS/SMB block size and other features of CIFS/SMB, and different O/S versions support different versions of the CIFS/SMB protocol.

Hence you might want to see if you can find a newer version of this MS Tech Note about CIFS/SMB block size tuning, and maybe some related ones on more general CIFS/SMB tuning.

However, I wouldn't expect miracles. At 40-60ms RTT with older versions of CIFS/SMB you need to start looking at WAN acceleration to get reasonable performance, and WAN acceleration is generally only available in enterprise grade products - I'm not aware of any free / OSS alternatives.

too high file I/O activity - weak performance - in foobar2k

Reply #6 –
Quote from: jazzmn on
hello there,

so here's the thing: i've noticed an extremely high file I/O activity in foobar2k. this was seen using process monitor (http://technet.microsoft.com/en-us/sysinte...s/bb896645.aspx) capable of logging everything a process does regarding file/registry OS calls.

I don't think things are as simple as that.  ProcMon is just logging the calls to the OS, not the OS's calls to the disk.

When I use Microsoft's Resource Monitor large chunks of the file are read fast and then nothing for minutes in my case.  (You need to look at the file being played, not just the foobar2000 task.)  I'm pretty sure my results are a little skewed by using the WaveForm Seekbar, but it's obvious that a lot of caching is going on.

too high file I/O activity - weak performance - in foobar2k

Reply #7 –
Quote from: jazzmn on
also, if foobar2000 starts rescanning the remote drive placed in media library, the flac playback is obviously a complete disaster.


This is perhaps related to the problem that made me turn off library monitoring a couple years ago.  Sometime after 1.x, a change was made which made foobar2000 lock up on all my computers if any I/O was done outside of foobar2000 in the library folders.  I now manually click a button to rescan my folders.

Have you tried using "full file buffering up to (kB)" in Advanced / Playback?

too high file I/O activity - weak performance - in foobar2k

Reply #8 –
Ouroboros,
surely, the exact type of the remote drive, OS details, what protocols are actually used is important, but it is not very relevant in this case and i'll explain immediately why.
tedsmith,
yes, procmon indeed logs calls made by processes to OS API, how does the OS handle those, when they'll actually be executed and how it's OS's job. however, exactly those calls from foobar are interesting for me because they talk about how foobar2k's file read code is written.

i've made a test. i've copied one mp3 file and one flac file from remote drive to local drive and played them locally while watching what happens in procmon AND resource monitor. the test reveals the following, as expected:
- mp3 files are read in chunks of ~16kb, thus in one second a regular medium quality mp3 file needs about 1-2 file reads/s, which is confirmed by procmon timestamps;
- flac files are read in chunks of ~8kb + an associated QueryStandardInformationFile command generating (depending on current flac bitrate) 15+ file reads/s and 15+ QueryStandardInformationFile/s (that actually return the same result always);
- the read pattern is exactly the same no matter if the file is read from local hdd or network drive so the fact that foobar2k accesses the file in small little chunks has nothing to do with CIFS/SMB; indeed, a remote drive is a remote drive and has latency and the communication between hosts may be optimized to reduce the latency, but the problem here is the abnormally high i/o reads/s generated by foobar2k;
- while playing, at least the first time, resource monitor CONFIRMS what we see on procmon, more precisely the mp3 is read with a constant speed of ~28kb/s in my case and the flac is read at ~130kb/s; of course, during next plays of the same files in many cases those are actually cached in memory so you see no activity in resource monitor for that file, although foobar2k accesses the files with the same pattern;
- when copying the files from network drive to local, using total commander, they are copied at ~1-1.5mb/s and procmon shows that total commander uses 1mb read chunks, generating a little more than one read command/s.

do you see the problem here? same conditions, same drive, total commander reads with 1mb/s+, foobar2k has difficulties passing over 100kb/s and playback of flac files usually drops. the difference can be seen in procmon, totalcmd uses 1mb read chunks, foobar2k does it with 8k. again: on a low latency drive this is no big deal, but on high latency i'm doomed.
this is a problem and it surely can be easily fixed by optimizing a bit the way foobar2k accesses the currently playing file. the code fix seems as simple as 'access the drive in 512k/1mb chunks or make this setting configurable'.

edit:
forgot to mention what i am actually using as my setup, although as i've said, local playback demonstrates that for foobar2k this doesn't matter and timings demonstrate that communication latency is as expected:
i am using sftp drive to mount to a network drive in windows OS a unix path from a remote server.
server runs debian squeeze (6), 'share' is stored on ext4 with noatime flag set.
a debian user is configured to have its home mapped to my 'share' and shell is set to scponly.
sftp drive free emulates and 'mounts' 'share' contents to a local network drive and handles server access through sftp protocol.
currently accessing 'share' from windows 8 pro and windows server 2012 machines.
as i've said, it is not blazing fast, can't be, it's pretty complicated, only about 1-2mb/s transfer rates in total commander but for music listening it would be enough if the number of high latency read requests would be reduced to a minimum in foobar2k.
the great advantage is that no matter where i go or i am, just install foobar and sftp drive free and can access and listen my entire collection like locally over a secure connection. only internet required.

too high file I/O activity - weak performance - in foobar2k

Reply #9 –
Quote from: neothe0ne on
Quote from: jazzmn on
also, if foobar2000 starts rescanning the remote drive placed in media library, the flac playback is obviously a complete disaster.


This is perhaps related to the problem that made me turn off library monitoring a couple years ago.  Sometime after 1.x, a change was made which made foobar2000 lock up on all my computers if any I/O was done outside of foobar2000 in the library folders.  I now manually click a button to rescan my folders.

Have you tried using "full file buffering up to (kB)" in Advanced / Playback?


doesn't work. set it to 2048kb, hit play, started almost immediately and after 4-5s playback started to drop. stopped at about 9s, procmon shows last read made at offset ~1.2m. so no, not taken into account in this case.

too high file I/O activity - weak performance - in foobar2k

Reply #10 –
Quote from: jazzmn on
Ouroboros,
surely, the exact type of the remote drive, OS details, what protocols are actually used is important, but it is not very relevant in this case and i'll explain immediately why.
tedsmith,
yes, procmon indeed logs calls made by processes to OS API, how does the OS handle those, when they'll actually be executed and how it's OS's job. however, exactly those calls from foobar are interesting for me because they talk about how foobar2k's file read code is written.

With no condescension intended: With an accurate description of the problem the developers can find the real source of the problem and address it.  Posting potential solutions only hides your problem in a sea of (possible) irrelevancies and lower's it's chances of being taken seriously.

How foobar2000's read code is written may have little to do with what you are seeing.  When playing, Foobar doesn't need to read any faster than the bits play.  The OS buffers just fine.  WaveForm Seekbar uses the same block size as the Foobar2000 play engine (heck probably the very same code) and it reads the files much faster.  You may indeed have a problem, but you are probably barking up the wrong tree with potential solutions.  As Knuth said  "Premature optimization is the root of all evil".

Anyway I'm bowing out now.

too high file I/O activity - weak performance - in foobar2k

Reply #11 –
Quote from: jazzmn on
- the read pattern is exactly the same no matter if the file is read from local hdd or network drive so the fact that foobar2k accesses the file in small little chunks has nothing to do with CIFS/SMB


Yes, and that's what you'd expect (the app shouldn't care about local vs. remote drives), and normally it's the combination of the network protocol and the RTT that then makes the remote access so slow. You are correct - under normal circumstances larger block sizes and fewer reads should improve performance. However, you aren't using CIFS/SMB over the network, you are using CIFS/SMB locally to talk to the SFTP client (which must be acting as an SMB server), then using SFTP across the network to actually transfer the file.

So, while tweaking the block size that foobar2000 reads may improve things slightly (if it is possible at all), I'd be more inclined to look at how the SFTP client and server are configured. The advantage is that's something you can do...

too high file I/O activity - weak performance - in foobar2k

Reply #12 –
Quote from: jazzmn on
Quote from: neothe0ne on
Quote from: jazzmn on
also, if foobar2000 starts rescanning the remote drive placed in media library, the flac playback is obviously a complete disaster.


This is perhaps related to the problem that made me turn off library monitoring a couple years ago.  Sometime after 1.x, a change was made which made foobar2000 lock up on all my computers if any I/O was done outside of foobar2000 in the library folders.  I now manually click a button to rescan my folders.

Have you tried using "full file buffering up to (kB)" in Advanced / Playback?


doesn't work. set it to 2048kb, hit play, started almost immediately and after 4-5s playback started to drop. stopped at about 9s, procmon shows last read made at offset ~1.2m. so no, not taken into account in this case.


Is your music less than 2 MB?  Full file buffering should only work if you set it to be greater than the size of your music... (e.g., I have it set to 14000 KB for my lossy encodes)

too high file I/O activity - weak performance - in foobar2k

Reply #13 –
wow, indeed, i wasn't paying attention to option name. set it to 50mb now and things changed a lot. when playing the file, foobar waits until the entire file is loaded into memory ('starting playback...'). what's the most interesting part is that in this case it actually reads in 1mb chunks and file is read from remote drive at the same speed as total commander would copy it. foobar and totalcmd generate in this case pretty much the same access pattern and transfer speed.
the bad part is that for a large flac it may take a while to start.

tedsmith, Ouroboros,
thanks much for your input, i appreciate it. it is not necessarily relevant but i am a developer with some years of experience and probably that's why i started to assume where the problem may be analyzing foobar's behaviour with tools, given that i don't have access to source code. but of course, i may be wrong.
let me rephrase a bit: the actual problem IS SFTP+CIFS/SMB+50ms-server and i'm aware of that. also, sftp drive probably doesn't optimize very well these small disk accesses in terms of read-aheads/buffered reads, but that's another story and i don't think i can do anything regarding that.
but: from my experience and logic, based on analysis results and comparison with totalcmd i *assume* that tweaking a bit the read buffers in foobar in some cases WILL overcome the setup's connection data transfer limitation. foobar can and knows how to read the file faster with much larger buffers, see above. and if it is possible for the file to be read by foobar fast (maybe 30s in-memory load time for a 5-6m remote flac in my case), this is a clear proof that some optimizations can be done that WILL solve my issue. in this case, actually foobar read in-memory the flac 10x faster than required playback speed while playing normally, with full file buffering off it merely reaches the required 1x speed.
that's why i believe that real optimizations exist and i pledge for larger or 'configurable' read buffers in foobar, but of course i don't know the code and implementation and it may not be that simple. in only assume it may be.
i didn't want to be 'smart' here at all, just want to help if foobar devs read this and point exactly to what i think is the issue as a dev-to-dev information. it is pretty much not useful to just claim that "playing from my remote drive doesn't work", i wanted to offer as much help as i can as a head-start.
however: of course, am willing to provide all required (technical) data to foobar's developers, just tell me what it is needed more precisely. i could even take a look over source code and try to patch it if i would have access.

another curiosity that still remains is why mp3 and flac access patterns are different? they both look sequential during playback, still, mp3 is accessed in 16k chunks, flac in 8k + QueryStandardInformationFile command..

thanks everyone.

too high file I/O activity - weak performance - in foobar2k

Reply #14 –
I hope that in my case new mainboard will solve the problem without necessity of waiting for full file buffering