New ABX blind-testing software
« on: April 21, 2014, 08:39:09 PM »
Hi there -- I'm new here, so if this is an inappropriate post, mods please accept my apologies and delete.

I wrote a new cross-platform ABX blind testing program. It also does shootouts, etc. It's free.

You can check it out here: http://lacinato.com/cm/software/othersoft/abx

I thought readers of this forum might find such a thing handy.

Thanks,
-c


tubestation

Re: New ABX blind-testing software
« Reply #1 on: April 29, 2014, 12:10:10 PM »
Hello clepsydrae !
Thank You very much for sharing this nice and usefull Tool !
I am going to install it to my Studio Pc to compare Mixes and Masters and also to listen to different modding Stages i do when fiddling on my Gear. It s also great  when you don t know what you are listening to and just let your ear judge !!!
Greatings,
Lothar.
« Last Edit: May 28, 2015, 09:08:37 PM by tubestation »

joaquins

Re: New ABX blind-testing software
« Reply #2 on: June 11, 2014, 08:27:30 PM »
Thanks for that, I'll install it when I get home...

JS
If I don't know how it works, I prefer don't turn it on.

Re: New ABX blind-testing software
« Reply #3 on: September 19, 2014, 04:03:18 AM »
New version just released! 2.3: Adds a bunch of features.

http://lacinato.com/cm/software/othersoft/abx

Thanks, -c

gyraf

Re: New ABX blind-testing software
« Reply #4 on: September 19, 2014, 05:46:29 AM »
Very, VERY interesting piece of software. Missed it the first time around, good that it was bumped..

Thanks!

Jakob E.
..note to self: don't let Harman run your company..

ruairioflaherty

Re: New ABX blind-testing software
« Reply #5 on: September 24, 2014, 03:01:55 AM »
This could be useful if it forced the audio interface to switch sample rates (ala Wavelab for example) rather than relying on OS SRC (which on a Mac colors the sound enough to make accurate comparisons impossible IMO).
 


Re: New ABX blind-testing software
« Reply #6 on: September 24, 2014, 03:31:40 AM »
Yeah, unfortunately, sample rate changing is out of the scope of this software (and all other ABX comparators that I'm aware of). It's a whole other level of sophistication that is way beyond my investment in an unpaid project. :-)

Personally I'd be comfortable setting the OS to the higher rate and letting it upsample from a lower rate for any such files, but it's obviously no longer "scientifically" accurate at that point. It's too bad that it's so involved to change the sample rate with sound cards, because it's one of the prime audio issues that people debate, but so it goes.

I bet some enterprising audio company could make the end-all ABX comparator software that does all that and more and sell it for $50 and make a little money. Then again, my experience with my own software is that people don't seem to really want to know the answers that it provides. :-)

But you can still use it to compare a whole hosts of other issues, like mic choice, mix A vs mix B, various compression parameters, etc etc.

ruairioflaherty

Re: New ABX blind-testing software
« Reply #7 on: September 25, 2014, 11:02:01 PM »
Yeah, unfortunately, sample rate changing is out of the scope of this software (and all other ABX comparators that I'm aware of). It's a whole other level of sophistication that is way beyond my investment in an unpaid project. :-)

Understood!  What makes it so complicated?   I've never understood why certain apps default to using the OS (with always on SRC) and others can force an external interface to change sample rate.  It seems like something I should know.

Quote
Personally I'd be comfortable setting the OS to the higher rate and letting it upsample from a lower rate for any such files, but it's obviously no longer "scientifically" accurate at that point. It's too bad that it's so involved to change the sample rate with sound cards, because it's one of the prime audio issues that people debate, but so it goes.

Come to my mastering room and listen with me to a level matched comparison between what the OS does to audio and what playing through a native interface does (I've tested this on every version of Mac OS since system 9).  Once you hear that you would lose all faith in any tests performed using the OS SRC.


I only need to use ABX testing when I'm trying to resolve small differences, IMO the OS SRC washes out those details and unfortunately it's an ever moving target. 

Quote
I bet some enterprising audio company could make the end-all ABX comparator software that does all that and more and sell it for $50 and make a little money. Then again, my experience with my own software is that people don't seem to really want to know the answers that it provides. :-)

Wanna start a Kickstarter?  I'll be the first to pledge $50 for your effort!  And yes, real testing often reveals things that are less than exciting - "You mean this $6,000 compressor sounds like ass when I actually level match it to the source!!"

Quote
But you can still use it to compare a whole hosts of other issues, like mic choice, mix A vs mix B, various compression parameters, etc etc.

Very true and I certainly don't mean to discourage, my needs are very specific in mastering (comparing SRC algorithms etc).

Re: New ABX blind-testing software
« Reply #8 on: September 26, 2014, 02:43:33 AM »
Yeah, unfortunately, sample rate changing is out of the scope of this software (and all other ABX comparators that I'm aware of). It's a whole other level of sophistication that is way beyond my investment in an unpaid project. :-)

Understood!  What makes it so complicated?   I've never understood why certain apps default to using the OS (with always on SRC) and others can force an external interface to change sample rate.  It seems like something I should know.

Likewise. I believe the upshot is that the OS only provides a very basic API to the sound interface. I.e. that there apparently isn't a standardized API for changing sample rates (which is weird to me, but presumably 16/44.1 was so universal it just didn't seem important.) So AFAIK if you want to change the sample rate you have to use the ASIO system (or DirectSound, or whatever), and that's a another order of complexity altogether, especially if you want to be cross-platform. In Java, and presumably in C++/etc., there are convenient libraries to simply load an audio clip and do clip.play(), whereas ASIO is more involved, so that would be my guess why many only more sophisticated programs handle the sample rate changes. (Although I only ever program in Java, which is higher-level and thus likely more limited than whatever low-level C libraries are around for specific operating systems, so maybe it's not so hard after all.)

That said, I just found a java ASIO library which might at least do the trick for windows platforms... if I ever receive more than $5 in donations i'll contemplate doing it some day, but it's not my focus right now. :-)

Further complicating the matter is that many interfaces don't immediately switch from one sample rate to another, and make little sounds and so forth when they do, which can reveal info about the audio being tested. Maybe the ABX software could always force a sample rate change to an unused sample rate in between playback of different files, just to obscure this.

Quote
Come to my mastering room and listen with me to a level matched comparison between what the OS does to audio and what playing through a native interface does (I've tested this on every version of Mac OS since system 9).  Once you hear that you would lose all faith in any tests performed using the OS SRC.

Sure, where's your mastering room? :-)

What exactly are you comparing with this test, and how? Audio at a certain sample rate being converted by the OS vs. audio being converted by the DAW before going to the interface? I'm a little confused...

Quote
Wanna start a Kickstarter?  I'll be the first to pledge $50 for your effort!

Thanks! Good to know. If I ever am seeking a new career, I'll consider it. :-)

joaquins

Re: New ABX blind-testing software
« Reply #9 on: September 26, 2014, 10:17:37 PM »
this web page has an AB no blind comparison, do you see it useful, maybe they are little biased, level comparison is waaay of, how this could be useful at all?

http://www.doctormix.com
If I don't know how it works, I prefer don't turn it on.


Re: New ABX blind-testing software
« Reply #10 on: September 26, 2014, 11:07:50 PM »
this web page has an AB no blind comparison, do you see it useful, maybe they are little biased, level comparison is waaay of, how this could be useful at all?

It could be useful for getting them business. :-) Yeah the levels are very different, obviously not a fair comparison, unless they would argue that increasing the level is part of their services. :-)

But I think there is a lot of use for non-blind AB comparisons... sometimes I use my own software when I just want a convenient way to level match files and to switch back and forth between them when they are playing. If you think you can hear a subtle difference, then you can go further with a blind test or a shootout.

Andy Peters

Re: New ABX blind-testing software
« Reply #11 on: September 27, 2014, 12:15:27 AM »
Understood!  What makes it so complicated?   I've never understood why certain apps default to using the OS (with always on SRC) and others can force an external interface to change sample rate.  It seems like something I should know.

Why? Because the applications designed for general use by consumers don't expect the consumer to have the first clue about any of this. The hardware informs the operating system which sample rates and word lengths are supported. In Mac OS X, the user can choose something in the Audio MIDI Setup. (In Windows, I have no idea any more.) So whatever choice is made there is the default. Also, most users have only one audio device connected to their computer. Certainly, based on the source's sample rate, whatever program that is playing audio could switch the hardware to match that sample rate -- if the hardware supports it. But the ugly truth is that most consumers can't distinguish between something played with SRC and something else played without it.

And the OS's audio system needs to be able to play streams from multiple sources at a time. Maybe Joe User is playing something back from iTunes, and he also wants the Mail program to ping when mail arrives, and he wants to hear the Facebook notifications, and so forth. All of these sorts of programs just send their sounds to the audio subsystem. The audio subsystem then SRCs each of these streams to the default output device's sample rate, and mixes them all together.

Remember -- that's how the average consumer user expects this all to happen.

Imagine the sort of chaos that could happen if iTunes was smart enough to change the hardware sample rate to match the various files' rates. (And my iTunes library is full of stuff ripped from CDs at 44.1 kHz/16-bit as well as some hi-res stuff from HD Tracks, and some other stuff I recorded and mixed that's at 48 kHz/24-bit.) So now something is playing along at 96 kHz and the mail tone comes through ... what should happen?

On the other hand, your favorite professional applications, liked Logic, query the audio driver to find out what sample rates the hardware supports, and allow you to decide what to use. This is because professional users of these programs know what they're doing and need this control.

Logic doesn't SRC (unless you use a plug-in to do that). If you have audio files recorded at 44.1 kHz and you tell the hardware to run at 48 kHz, it will sound wrong.

-a
"On the Internet, nobody can hear you mix a band"

Re: New ABX blind-testing software
« Reply #12 on: September 30, 2014, 12:16:26 AM »
2.32 adds support for drag/drop of *.abx files and opening *.abx files directly (once they are associated in the OS to "open with" the program.)

As always, latest at: http://lacinato.com/cm/software/othersoft/abx

Re: New ABX blind-testing software
« Reply #13 on: October 10, 2014, 04:44:47 PM »
Lots of little UI fixes and improvements for 2.34:

http://lacinato.com/cm/software/othersoft/abx

Re: New ABX blind-testing software
« Reply #14 on: July 21, 2015, 01:28:22 AM »
Version 2.35 posted -- fixes a lockup/freeze some Mac OS X users were experiencing.

Re: New ABX blind-testing software
« Reply #15 on: December 24, 2015, 03:39:41 AM »
Version 2.36 released --

* in shootouts you can now optionally score each file 1-10 as you go
* auto-detect changes to audio files and prompt to refresh/ignore (except on Windows due to Java bug)
* keyboard shortcuts
* prompts to prevent accidental loss of test results
* lots of little UI improvements
« Last Edit: December 24, 2015, 03:46:57 AM by clepsydrae »

Re: New ABX blind-testing software
« Reply #16 on: December 25, 2015, 04:21:00 AM »
(also I have just released a new web-based ABX/Shootout tool -- see the website for more!)

Re: New ABX blind-testing software
« Reply #17 on: June 26, 2017, 04:16:27 AM »
Thank you! I'll try this tool in my studio  https://smartmixing.com/

JohnRoberts

Re: New ABX blind-testing software
« Reply #18 on: June 26, 2017, 10:07:03 AM »
Does software based ABX include any of the statistical analysis tools?  That seems a natural for software based app.

JR
Don't only half-ass tune your drums. Visit https://circularscience.com to hear what properly "cleared" drums sound like.

Re: New ABX blind-testing software
« Reply #19 on: June 26, 2017, 12:43:50 PM »
You mean like computing LUFS and the like?