Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: peak level batch analysis (Read 33673 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

peak level batch analysis

Hi there.

I need a software, that is capable to do:

1) batch normalize (peak level) on several files.
2) generate log file with normalize results.

the general purpose is to get the "peak level" (not RMS, peak only) with 0.1dB accuracy (values must be in decibels nor precents) of several audio WAV files (6000 to 20 000 files), and to easy transform it into the excel or similar sheet for later analysis.

I don't know if there is any (user friendly) signal batch processing software for audio, that could do that, but "normalizers" can do it (for example wavtrim). The only problem is to get the log data from this processing in plain text (wavtrim displays the results, but does not creates the text reports; you see these values but have to rewrite them).

*

Another useful thing would be an analyser (also with batch mode) capable to find automatically multiple "peaks" (frequencies) over the background noise in FFT analysis (524288 pts accuracy, not 64k which is too small). In general I need to know the frequencies of several peaks in desired frequency range, peaks with 0,1Hz to 100Hz difference, and also in multiple files/ranges.

*

could somebody help? rewriting it manually is horrible.

p.s.: I'm Windows (XP) user, not familiar with programming or scripting (like matlab) languages.

peak level batch analysis

Reply #1 –
I don't know if it will meet your needs, but check into SoX[/color].

Quote
p.s.: I'm Windows (XP) user, not familiar with programming or scripting (like matlab) languages.
  Of course, repetitive mathematical tasks with specific output requirements are exactly the kinds of things that computers/programs are good for.  I've never used Matlab, but this looks like a perfect Matlab application!

Quote
Another useful thing would be an analyser (also with batch mode) capable to find automatically multiple "peaks" (frequencies) over the background noise in FFT analysis...
Normalizing & finding/recording amplitude peaks is straight-forward, but I'm pretty sure it's going to take some custom programming to find peaks in certain frequency bands and compare those to background noise...


Quote
....several audio WAV files (6000 to 20 000 files),
  "Several", you could do one at a time "by hand".  Several thousand calls for automation!

peak level batch analysis

Reply #2 –
Do you need intersample peak measurement?

peak level batch analysis

Reply #3 –
Quote from: Axon on
Do you need intersample peak measurement?


everything is in WAV files, 44.1kHz -- 16bit -- mono. no additional transformations of sound files. no complicated analysis of peak-related levels.

in general the source files have multi-band peak events (something between A to B Hz, something between C to D Hz, and so on), and those events are compared with each other in 4 step analysis (for example t1=A msec, t2=B sec, t3=C sec, and so on). in a graph ypu get n-point curves (8 to 12 bands per source). the source files are also split into the "freq band related" pieces and then into "time steps" for "easy listening" tests (to find out which data is important, and which is not).

*

Quote from: DVDdoug link=msg=0 date=
I don't know if it will meet your needs, but check into SoX.


I see no gui for windows. how to setup batch normalizing (or just peak measurement) with log report what was done (and what values in dB were detected in sounrce files)?

peak level batch analysis

Reply #4 –
Quote from: krjrs on
I need a software, that is capable to do:

1) batch normalize (peak level) on several files.
2) generate log file with normalize results.

the general purpose is to get the "peak level" (not RMS, peak only) with 0.1dB accuracy (values must be in decibels nor precents) of several audio WAV files (6000 to 20 000 files), and to easy transform it into the excel or similar sheet for later analysis.

If you don't fear the command line and a bit shell programming you may give the following script a try:

Code: [Select]
#!/bin/sh
# usage: sh <name of script> <source dir> <target dir>

IFS="
"

source="${1}"
target="${2}"
protocol="protocol.csv"

rm -f "${protocol}"
echo "file;scale" > "${protocol}"

for i in `find "${source}" -name "*.wav"`; do
  echo "processing \"${i}\" ..."
  offs=`expr index "${i}" "/"`
  offs=`expr ${offs} + 1`
  length=`expr length "${i}"`
  length=`expr 1 + ${length} - ${offs}`
  j="${target}/`expr substr "${i}" ${offs} ${length}`"
  mkdir -p "`dirname "${j}"`"
  q=`sox "${i}" "${j}" stat gain -n 2>&1\
      |grep "Volume adjustment"\
      |sed "s/^[^0-9.]*\([0-9.]*\)/\1/g"`
  echo "${i};${q}" >> "${protocol}"
done

The script utiliizes the SoX "stat" and "gain -n" effects (cf. SoX man page) and picks the scale factor from the ouput. The script writes the scaling factors per processed file into "protocol.csv".

The script is executed by running the following command from a MSys shell:

Code: [Select]
sh norm.sh <input dir> <output dir>

peak level batch analysis

Reply #5 –
Quote from: pbelkner on
If you don't fear the command line and a bit shell programming you may give the following script a try:


I did following.

1. SoX files installed in directory: "C:\sox-14-3-1\"
2. in the same directory of SoX - created file "norm.sh" with the content you have placed in code above
3. placed some (44k/16bit/mono) WAV files (01.wav, 02.wav,...) to input directory "L:\_audio_\test\"
4. created output directory "L:\_audio_\test2\"
5 started the winshell and put following:

C:\sox-14-3-1\sox.exe norm.sh L:\_audio_\test L:\_audio_\test2

and then entered OK.
a small DOS window appeared and dissapeared.
no new files (or "protocol.csv" file) in any of these 3 directories (SoX, input, output).

also nothing when tried with:

C:\sox-14-3-1\sox.exe sh norm.sh L:\_audio_\test L:\_audio_\test2

*.sh files are not associated with SoX.

What did I incorrect?

peak level batch analysis

Reply #6 –
.shell scripts are lists of instructions to be executed by *n*x-like shells such as BASH--not by Windows, much less by Windows programs such as SoX.

There may be a way to translate that script for Windows' own shell (cmd.exe in XP, PowerShell in Vista and later), but I wouldn't know where to start, and I imagine that it varies with the version of Windows you're using.

peak level batch analysis

Reply #7 –
You need a unix-style shell interpreter for windows for that script, as it's a unix shell script. pbelkner specified the MSys shell, but other unix shell interpreters for windows should work as well.

Otherwise you need to write the script in another scripting/programming language you have an interpreter/compiler for.

peak level batch analysis

Reply #8 –
Quote from: krjrs on
5 started the winshell and put following:

C:\sox-14-3-1\sox.exe norm.sh L:\_audio_\test L:\_audio_\test2

and then entered OK.

This script needs MSys to be installed:

Quote from: pbelkner on
The script is executed by running the following command from a MSys shell:

Code: [Select]
sh norm.sh <input dir> <output dir>

Try the following:
  • Download the MSys installer.
  • Run the MSys installer. Answer the questions as follows:[blockquote]This is a post install process that will try to normalize between
    your MinGW install if any as well as your previous MSYS installs
    if any.  I don't have any traps as aborts will not hurt anything.
    Do you wish to continue with the post install? [yn ] y

    Do you have MinGW installed? [yn ] n[/blockquote]For the rest hit enter.
  • Copy
    Code: [Select]
    #Win32_Path        Mount_Point
    C:/                /c/
    L:/                /l/
    as a file named "fstab" to "C:\msys\etc" (there you should already find a file named "fstab.sample"). You may add more lines according to your DOS drives.
  • Install SoX to "C:\msys\local\bin" (or likewise to "C:\msys\bin").
  • Start the MSys shell, i.e. from Windows Explorer double click "C:\msys\msys.bat". Please note that in the MSys shell you have UNIX forward slashs "/" instead of DOS backslashs.
  • In MSys shell cd to your working directory, i.e. to the directory where the script "norm.sh" is located.
    Code: [Select]
    cd /c/working/
  • Run the script "norm.sh".
    Code: [Select]
    sh norm.sh /l/_audio_/test /l/_audio_/test2

There is a fine point I didn't know how to deal with. If you copy the script "norm.sh" from above and save it using e.g. "notepad" you will propably end up with DOS like "carrige return/line feed" at the end of lines. What is needed is UNIX like "line feed" only. You may resolve this using gvim (is not everybodys friend).

The script's original version has problems with an absolute path. Following is a corrected version:

Code: [Select]
#!/bin/sh
# usage: sh <name of script> <source dir> <target dir>

IFS="
"

source="${1}"
target="${2}"
protocol="`pwd`/protocol.csv"

rm -f "${protocol}"
echo "file;scale" > "${protocol}"

cd ${source}

for i in `find "./" -name "*.wav"`; do
  echo "processing \"${i}\" ..."
  j="${target}/${i}"
  mkdir -p "`dirname "${j}"`"
  q=`sox "${i}" "${j}" stat gain -n 2>&1\
      |grep "Volume adjustment"\
      |sed "s/^[^0-9.]*\([0-9.]*\)/\1/g"`
  echo "${i};${q}" >> "${protocol}"
done

peak level batch analysis

Reply #9 –
Quote from: krjrs on
Quote from: Axon on
Do you need intersample peak measurement?


everything is in WAV files, 44.1kHz -- 16bit -- mono. no additional transformations of sound files. no complicated analysis of peak-related levels.

in general the source files have multi-band peak events (something between A to B Hz, something between C to D Hz, and so on), and those events are compared with each other in 4 step analysis (for example t1=A msec, t2=B sec, t3=C sec, and so on). in a graph ypu get n-point curves (8 to 12 bands per source). the source files are also split into the "freq band related" pieces and then into "time steps" for "easy listening" tests (to find out which data is important, and which is not).
Can you explain exactly what you're trying to do? It sounds like some kind of psychoacoustic experiment - but a typical approach would be to generate the audio files yourself to contain exactly the test stimuli that you wanted - not to start with some audio files and try to figure out what's in them.

If the nature of the experiment relies on these pre-existing audio files, then the specific analysis you carry out will have a direct impact on the final results.

I can't imagine, from what you've said, that simple absolute peak level is much use. Also, your idea of using an FFT and checking the level in the frequency band with the most energy (if implemented that simply) is very unlikely to give you what you want (whatever it is that you want!).

Cheers,
David.

peak level batch analysis

Reply #10 –
Quote from: pbelkner on
This script needs MSys to be installed:

(...)


I was able to do everything what you wrote.
the script code was copy/paste via gvim.

after I started the script, the mingw32 window displayed "processing" lines (like [processing "./01.wav" ...] and so on) for each file from the source dir. "protocol.csv" has been created. no output files has been created. input files are also unchanged (about -20dB as before). header of csv file contains 2 columns (file;scale), but the data contains only the column with filenames (the second column is empty).

what I'm missing?

*

@2Bdecided: I'm sory, but I can't explain it in details. It is required for statistical + comparative multiband analysis with listening reference.
 

peak level batch analysis

Reply #11 –
Quote from: krjrs on
I was able to do everything what you wrote.

That's good news.

Quote from: krjrs on
input files are also unchanged (about -20dB as before).

That's by intention. The output should be in the directory tree you've specified on the command line as <output dir>.

Quote from: krjrs on
no output files has been created.

Quote from: krjrs on
header of csv file contains 2 columns (file;scale), but the data contains only the column with filenames (the second column is empty).

This seems to indicate that the "sox" command failed. What happens if you simply type "sox" in the MSys shell? Is "sox" available?

Below you find a new version of the script allowing more verbose output by providing an optional third parameter:

Code: [Select]
sh norm.sh <input dir> <output dir> verbose

Running the script this way will allow you to see error messages helping us to identify what's causing the problem.

Code: [Select]
#!/bin/sh
# usage: sh <name of script> <source dir> <target dir>

IFS="
"

source="${1}"
target="${2}"
verbose="${3}"
protocol="`pwd`/protocol.csv"

rm -f "${protocol}"
echo "file;scale" > "${protocol}"

cd ${source}

for i in `find "./" -name "*.wav"`; do
  j="${target}/${i}"
  echo "processing \"${i}\" to \"${j}\" ..."
  mkdir -p "`dirname "${j}"`"
  out=`sox "${i}" "${j}" stat gain -n 2>&1`

  if [ ${verbose} ]; then
    echo "${out}"
  fi

  q=`echo "${out}"|sed -n "s/^Volume adjustment:[ ]*\([0-9.]*\)/\1/p"`
  echo "${i};${q}" >> "${protocol}"
done

peak level batch analysis

Reply #12 –
Quote from: pbelkner on
This seems to indicate that the "sox" command failed. What happens if you simply type "sox" in the MSys shell? Is "sox" available?
(...)
Running the script this way will allow you to see error messages helping us to identify what's causing the problem.


yes, sox seem to be available (sox help is displayed).

when I type "sh norm.sh /l/test1 /l/test2 verbose" (I have simplified the i/o paths), it goes like that:

processing "./01.wav" to "/l/test2/./01.wav" ...
C:\msys\1.0\bin\sox.exe: SoX v14.3.1
[...the help of sox goes...]
FAIL sox: Not enough input filenames specified

for each file similar.

it recognizes input filenames in source folder, but have some problem with input.

peak level batch analysis

Reply #13 –
Quote from: krjrs on
FAIL sox: Not enough input filenames specified

This is a strange error.

Strange errors in shell scripts are often due to DOS end of lines.

Please don't forget ":set notextmode" when gvim is in command mode before saving. After saving gvim should display that the file is in unix mode (last line, right to the filename).

peak level batch analysis

Reply #14 –
Quote from: pbelkner on
Please don't forget ":set notextmode" when gvim is in command mode before saving. After saving gvim should display that the file is in unix mode (last line, right to the filename).


hmm...? gvim seems to be rather like a strange notepad (in start menu I have gvim, gvim diff, gvim easy and some other items). there are file format settings in "edit" menu, where I can select "unix". did you mean that? but I have found other text editor, which converts the end-lines to "LF UNIX" (if this is what you mean). the same result.

did you checked this script on your computer?
can you send me the script file to e-mail?
krjrs "at" poczta "dot" onet "another-dot" pl ?

peak level batch analysis

Reply #15 –
Quote from: krjrs on
there are file format settings in "edit" menu, where I can select "unix". did you mean that?

Yep. Never saw this before.

Quote from: krjrs on
but I have found other text editor, which converts the end-lines to "LF UNIX" (if this is what you mean). the same result.

Then possibly something else causes the problem, I suspect.

From where you get SoX? I've used this one.

Quote from: krjrs on
did you checked this script on your computer?

Yes:

Code: [Select]
pbelkner@pc ~
$ sh norm.sh /d/X/source/ /d/X/target/ verbose
processing "./x-44100.wav" to "/d/X/target//./x-44100.wav" ...
Samples read:            424600
Length (seconds):      4.814059
Scaled by:         2147483647.0
Maximum amplitude:     0.541840
Minimum amplitude:    -0.525574
Midline amplitude:     0.008133
Mean    norm:          0.055085
Mean    amplitude:     0.000010
RMS     amplitude:     0.090156
Maximum delta:         0.607910
Minimum delta:         0.000000
Mean    delta:         0.062534
RMS     delta:         0.102108
Rough   frequency:         7949
Volume adjustment:        1.846
D:\X\msys\local\bin\sox.exe WARN dither: dither clipped 1 samples; decrease volume?

pbelkner@pc ~
$ _

Quote from: krjrs on
can you send me the script file to e-mail?

I've managed to upload it to the forum.

peak level batch analysis

Reply #16 –
Quote from: pbelkner on
Then possibly something else causes the problem, I suspect.

From where you get SoX? I've used this one.

I've managed to upload it to the forum.


Hmm... That's odd. I downloaded your script. I have the same version of SoX as you (used your links). I even moved the test folders to /c/ (I remember that some older dos software had drive-letter issues). The result is the same as before: "FAIL sox: Not enough input filenames specified" on each file. Either the SoX has some problems with WAV headers and can not read my WAV files (which are not unusual, eihter wavelab or dbpoweramp converted) or something is wrong with installation (in what way?), or... I don't know. NTFS not supported properly? I try this later on another computer.

peak level batch analysis

Reply #17 –
Are there any spaces in the filenames?

peak level batch analysis

Reply #18 –
Quote from: Nick.C on
Are there any spaces in the filenames?


no, I never use "spaces" in filenames. tested files were named with numbers and letters only.

peak level batch analysis

Reply #19 –
On the second computer (different winXP installation) the problem is the same.

Any ideas what could be wrong?

Any other alternatives to peak level detection with data logging through batch normalizing?

peak level batch analysis

Reply #20 –
Quote from: krjrs on
FAIL sox: Not enough input filenames specified

I've never sawn this error before. Meanwhile I've been able to reproduce it. You get this error if providing SoX with only one file name, corresponding to an existing file or not.

Code: [Select]
pbelkner@pc ~
$ sox /d/X/source/x-44100.wav
D:\X\msys\local\bin\sox.exe: SoX v14.3.1

Usage summary: [gopts] [[fopts] infile]... [fopts] outfile [effect [effopt]]...

...

D:\X\msys\local\bin\sox.exe FAIL sox: Not enough input filenames specified


pbelkner@pc ~
$ _

If it is the same if you run the script, variable "${j}" defined in line 18 should be empty. But then you should never see

Quote from: krjrs on
processing "./01.wav" to "/l/test2/./01.wav" ...

because this is of line 19 using "${j}".

What happens if you run the following command (it's essentialy line 21)?

Code: [Select]
sox "/l/test/01.wav" "/l/test2/./01.wav" stat gain -n 2>&1

If this works fine I suspect that something went wrong during copying the script.

peak level batch analysis

Reply #21 –
the same thing. after typing

sox "/c/test1/01.wav" "/c/test2/./01.wav" stat gain -n 2>&1

the sox starts, displays its help and then the error message:

C:\msys\1.0\bin\sox.exe FAIL sox: Not enough input filenames specified

something with default access paths?

peak level batch analysis

Reply #22 –
Quote from: krjrs on
the same thing. after typing

sox "/c/test1/01.wav" "/c/test2/./01.wav" stat gain -n 2>&1

the sox starts, displays its help and then the error message:

C:\msys\1.0\bin\sox.exe FAIL sox: Not enough input filenames specified

something with default access paths?

Ok, seems to be SoX and not the script. What's the result of "which sox"?

Code: [Select]
pbelkner@pc ~
$ which sox
/usr/local/bin/sox

pbelkner@pc ~
$ _

peak level batch analysis

Reply #23 –
Quote from: pbelkner on
Ok, seems to be SoX and not the script. What's the result of "which sox"?

Code: [Select]
pbelkner@pc ~
$ which sox
/usr/local/bin/sox

pbelkner@pc ~
$ _


krjrs@AYA ~
$ which sox
/bin/sox