New ABX blind-testing software

GroupDIY Audio Forum

Help Support GroupDIY Audio Forum:

This site may earn a commission from merchant affiliate links, including eBay, Amazon, and others.

clepsydrae

Active member
Joined
Apr 19, 2014
Messages
44
Hi there -- I'm new here, so if this is an inappropriate post, mods please accept my apologies and delete.

I wrote a new cross-platform ABX blind testing program. It also does shootouts, etc. It's free.

You can check it out here: http://lacinato.com/cm/software/othersoft/abx

I thought readers of this forum might find such a thing handy.

Thanks,
-c
 
Hello clepsydrae !
Thank You very much for sharing this nice and usefull Tool !
I am going to install it to my Studio Pc to compare Mixes and Masters and also to listen to different modding Stages i do when fiddling on my Gear. It s also great  when you don t know what you are listening to and just let your ear judge !!!
Greatings,
Lothar.
 
This could be useful if it forced the audio interface to switch sample rates (ala Wavelab for example) rather than relying on OS SRC (which on a Mac colors the sound enough to make accurate comparisons impossible IMO).
 

 
Yeah, unfortunately, sample rate changing is out of the scope of this software (and all other ABX comparators that I'm aware of). It's a whole other level of sophistication that is way beyond my investment in an unpaid project. :)

Personally I'd be comfortable setting the OS to the higher rate and letting it upsample from a lower rate for any such files, but it's obviously no longer "scientifically" accurate at that point. It's too bad that it's so involved to change the sample rate with sound cards, because it's one of the prime audio issues that people debate, but so it goes.

I bet some enterprising audio company could make the end-all ABX comparator software that does all that and more and sell it for $50 and make a little money. Then again, my experience with my own software is that people don't seem to really want to know the answers that it provides. :)

But you can still use it to compare a whole hosts of other issues, like mic choice, mix A vs mix B, various compression parameters, etc etc.
 
clepsydrae said:
Yeah, unfortunately, sample rate changing is out of the scope of this software (and all other ABX comparators that I'm aware of). It's a whole other level of sophistication that is way beyond my investment in an unpaid project. :)

Understood!  What makes it so complicated?  I've never understood why certain apps default to using the OS (with always on SRC) and others can force an external interface to change sample rate.  It seems like something I should know.

Personally I'd be comfortable setting the OS to the higher rate and letting it upsample from a lower rate for any such files, but it's obviously no longer "scientifically" accurate at that point. It's too bad that it's so involved to change the sample rate with sound cards, because it's one of the prime audio issues that people debate, but so it goes.

Come to my mastering room and listen with me to a level matched comparison between what the OS does to audio and what playing through a native interface does (I've tested this on every version of Mac OS since system 9).  Once you hear that you would lose all faith in any tests performed using the OS SRC.


I only need to use ABX testing when I'm trying to resolve small differences, IMO the OS SRC washes out those details and unfortunately it's an ever moving target. 

I bet some enterprising audio company could make the end-all ABX comparator software that does all that and more and sell it for $50 and make a little money. Then again, my experience with my own software is that people don't seem to really want to know the answers that it provides. :)

Wanna start a Kickstarter?  I'll be the first to pledge $50 for your effort!  And yes, real testing often reveals things that are less than exciting - "You mean this $6,000 compressor sounds like ass when I actually level match it to the source!!"

But you can still use it to compare a whole hosts of other issues, like mic choice, mix A vs mix B, various compression parameters, etc etc.

Very true and I certainly don't mean to discourage, my needs are very specific in mastering (comparing SRC algorithms etc).
 
ruairioflaherty said:
clepsydrae said:
Yeah, unfortunately, sample rate changing is out of the scope of this software (and all other ABX comparators that I'm aware of). It's a whole other level of sophistication that is way beyond my investment in an unpaid project. :)

Understood!  What makes it so complicated?  I've never understood why certain apps default to using the OS (with always on SRC) and others can force an external interface to change sample rate.  It seems like something I should know.

Likewise. I believe the upshot is that the OS only provides a very basic API to the sound interface. I.e. that there apparently isn't a standardized API for changing sample rates (which is weird to me, but presumably 16/44.1 was so universal it just didn't seem important.) So AFAIK if you want to change the sample rate you have to use the ASIO system (or DirectSound, or whatever), and that's a another order of complexity altogether, especially if you want to be cross-platform. In Java, and presumably in C++/etc., there are convenient libraries to simply load an audio clip and do clip.play(), whereas ASIO is more involved, so that would be my guess why many only more sophisticated programs handle the sample rate changes. (Although I only ever program in Java, which is higher-level and thus likely more limited than whatever low-level C libraries are around for specific operating systems, so maybe it's not so hard after all.)

That said, I just found a java ASIO library which might at least do the trick for windows platforms... if I ever receive more than $5 in donations i'll contemplate doing it some day, but it's not my focus right now. :)

Further complicating the matter is that many interfaces don't immediately switch from one sample rate to another, and make little sounds and so forth when they do, which can reveal info about the audio being tested. Maybe the ABX software could always force a sample rate change to an unused sample rate in between playback of different files, just to obscure this.

Come to my mastering room and listen with me to a level matched comparison between what the OS does to audio and what playing through a native interface does (I've tested this on every version of Mac OS since system 9).  Once you hear that you would lose all faith in any tests performed using the OS SRC.

Sure, where's your mastering room? :)

What exactly are you comparing with this test, and how? Audio at a certain sample rate being converted by the OS vs. audio being converted by the DAW before going to the interface? I'm a little confused...

Wanna start a Kickstarter?  I'll be the first to pledge $50 for your effort!

Thanks! Good to know. If I ever am seeking a new career, I'll consider it. :)
 
this web page has an AB no blind comparison, do you see it useful, maybe they are little biased, level comparison is waaay of, how this could be useful at all?

http://www.doctormix.com
 
joaquins said:
this web page has an AB no blind comparison, do you see it useful, maybe they are little biased, level comparison is waaay of, how this could be useful at all?

It could be useful for getting them business. :) Yeah the levels are very different, obviously not a fair comparison, unless they would argue that increasing the level is part of their services. :)

But I think there is a lot of use for non-blind AB comparisons... sometimes I use my own software when I just want a convenient way to level match files and to switch back and forth between them when they are playing. If you think you can hear a subtle difference, then you can go further with a blind test or a shootout.
 
ruairioflaherty said:
Understood!  What makes it so complicated?  I've never understood why certain apps default to using the OS (with always on SRC) and others can force an external interface to change sample rate.  It seems like something I should know.

Why? Because the applications designed for general use by consumers don't expect the consumer to have the first clue about any of this. The hardware informs the operating system which sample rates and word lengths are supported. In Mac OS X, the user can choose something in the Audio MIDI Setup. (In Windows, I have no idea any more.) So whatever choice is made there is the default. Also, most users have only one audio device connected to their computer. Certainly, based on the source's sample rate, whatever program that is playing audio could switch the hardware to match that sample rate -- if the hardware supports it. But the ugly truth is that most consumers can't distinguish between something played with SRC and something else played without it.

And the OS's audio system needs to be able to play streams from multiple sources at a time. Maybe Joe User is playing something back from iTunes, and he also wants the Mail program to ping when mail arrives, and he wants to hear the Facebook notifications, and so forth. All of these sorts of programs just send their sounds to the audio subsystem. The audio subsystem then SRCs each of these streams to the default output device's sample rate, and mixes them all together.

Remember -- that's how the average consumer user expects this all to happen.

Imagine the sort of chaos that could happen if iTunes was smart enough to change the hardware sample rate to match the various files' rates. (And my iTunes library is full of stuff ripped from CDs at 44.1 kHz/16-bit as well as some hi-res stuff from HD Tracks, and some other stuff I recorded and mixed that's at 48 kHz/24-bit.) So now something is playing along at 96 kHz and the mail tone comes through ... what should happen?

On the other hand, your favorite professional applications, liked Logic, query the audio driver to find out what sample rates the hardware supports, and allow you to decide what to use. This is because professional users of these programs know what they're doing and need this control.

Logic doesn't SRC (unless you use a plug-in to do that). If you have audio files recorded at 44.1 kHz and you tell the hardware to run at 48 kHz, it will sound wrong.

-a
 
2.32 adds support for drag/drop of *.abx files and opening *.abx files directly (once they are associated in the OS to "open with" the program.)

As always, latest at: http://lacinato.com/cm/software/othersoft/abx
 
Version 2.36 released --

* in shootouts you can now optionally score each file 1-10 as you go
* auto-detect changes to audio files and prompt to refresh/ignore (except on Windows due to Java bug)
* keyboard shortcuts
* prompts to prevent accidental loss of test results
* lots of little UI improvements
 

Latest posts

Back
Top