DIY compatible Waves Soundgrid Server

[GreyboxAudio]

Hey everyone,

so it's been a while and I got good news for everyone! You can pretty much ignore everything I wrote up to this point  ;D

The challenge:
You need to create an EFI bootable USB-Stick with syslinux.

If you're on Windows:
- Download Rufus from https://rufus.ie/
- Once open, toggle "Show advanced drive properties" (below the "Partition scheme"-Dropdown).
You don't need to click anything here, instead click on the "Boot selection"-Dropdown
-  select "Syslinux 6.04" from  the "Boot selection"-Dropdown
- Make sure you've selected the correct USB drive and hit START.
You'll get some warning dialogs, as well as some dialogs where you need to allow Rufus to download syslinux.

######

Once that's done:
- register at Waves and get a free license for Soundgrid Studio here https://www.waves.com/mixers-racks/soundgrid-studio-emotion-st-8-ch#introducing-the-new-waves-soundgrid-studio
- install Soundgrid Studio on your System
- if you're on Windows navigate to C:\ProgramData\Waves Audio\SoundGrid Firmware\SGS
-  on macOS the files are in /Library/⁨Application Support/Waves⁩ /SoundGrid Firmware⁩/⁨SGS⁩
- there you'll find the latest server firmware as a .wfi file
- make a copy to a folder of your choice
- use 7-zip (https://www.7-zip.org/) to simply extract the .wfi file to your bootable USB stick
- on macOS you can try keka or The Unarchiver instead
- boot up and have fun!

 

[GreyboxAudio]

I used a pre-formatted stick for this, which I created from a V11 server image someone shared with me. (That's why I can't give you a tutorial yet on how to create the syslinux partitioning from scratch).
I removed all the files on the stick, keeping the partitioning intact, and then simply extracted the .wfi file and everything booted up nicely.

The V11 release is much more compatible than V9. It even worked on a mainboard I had already given up on!
The source files on Waves' website are completely redundant now and my guess is, that they eventually stopped modifying the image as the tools & kernel evolved.

I also built a completely new server based on the Gigabyte B360N WiFi (or GSM, depending on your region), which is the same motherboard Waves are using for all their current servers.
The important thing to know about this board is, that you need to disable one of the two LAN controllers in the BIOS. It doesn't matter which one. You also need to connect a network cable at boot, or the system might get stuck in an initilization loop until you have a network connection.

Because I didn't want to take too much of a risk, I went with the cheapest compatible Intel CPU I could get: the Celeron G4930  ;D Going with a Core i3 right away would have been a 100€ increase.
For just 35€ the Celeron works actually quite well. If you heavily rely on H-Reverb or Abbey Road stuff, this won't do. But for small setups with mostly dynamics & eq it can handle as much as you can throw at it.

As for memory I just went with a single stick of 4GB DDR4 2400 from G.Skill. 4GB doesn't sound much in 2020 and if you'd be running Windows 10 with that, you'd  have a hard time. For a soundgrid server however, that's plenty. You can check the memory usage on your server by activating the console and entering cat /proc/meminfo
The OS itself only uses around 650mb and filling up all 128 inserts in SGS only increased memory usage by 200mb in my case.
I know memory is cheap right now, but investing $20 in good quality case fans will benefit you much more, than adding another 4GB of memory.

When shopping for a PSU something around 300W will do just fine and has enough headroom, even if you upgrade to an i9 9900k later on.

 

[Whoops]

The source files on Waves' website are completely redundant now

The Source Files were always redundant, I told that in the first post from February of 2019.

It seems the image of the soundgrid server software is included in the Servers  update software package.

And I reminded again when you got into an unnecessary extremely complicated endeavor.
Nice that you admit all of that should be ignored, to be honest personally I think you should delete your previous posts as they don't help this thread in any way. People that come in here looking for help, directions on how to build the server will just get really confused, there's no need for that.

Hello everyone,
thanks for yous posts in this thread.

The open source version it's not really needed, first of all it's not complete and second the complete soundgrid server image is available in the soundgrid studio update softwares. search inside appdata folder.

I have the Soundgrid server image and I can send them to any of you, please pm me

But for small setups with mostly dynamics & eq it can handle as much as you can throw at it.

To handle Dynamics and EQs no one needs a server...
Native runs great for that and without latency problems.

There's no point in building a soundgrid server that has no relevant processing power

 

[Whoops]

The challenge:
You need to create an EFI bootable USB-Stick with syslinux.
I haven't had the time to figure it out, but you can probably use any Linux distribution you want and just use the latest syslinux release.

######

Once that's done:
- register at Waves and get a free license for Soundgrid Studio here https://www.waves.com/mixers-racks/soundgrid-studio-emotion-st-8-ch#introducing-the-new-waves-soundgrid-studio
- install Soundgrid Studio on your System
- if you're on Windows navigate to C:\ProgramData\Waves Audio\SoundGrid Firmware\SGS
- [don't have a mac, so someone can maybe share the folder path here  )
- you'll find the latest server firmware as a .wfi file
- make a copy to a folder of your choice
- use 7-zip (https://www.7-zip.org/) to simply extract the .wfi file to your bootable USB stick
- [still don't have a mac  appreciate if someone can try this on mac]
- boot up and have fun!

Thank you so much for the post and for sharing that information and your experience
Thank you

 

[GreyboxAudio]

Thanks for your reply. Just to get this out of the way: I don't want to fight and I really appreciate all your support 

The Source Files were always redundant, I told that in the first post from February of 2019.

Yes and I knew from the start that those aren't the complete server image files. But there is a lot of confusion about the open source server software, so I wanted to see, what those files actually are and what can be done with them, because so far nobody has done that. And even though this looks now like a big waste of time,  I've learned a lot about linux and had some fun (and some frustration ... alright, lots! of frustration  ;D)

And I reminded again [about the server files] when you got into an unnecessary extremely complicated endeavor.

I know, this has popped up a couple of times. But you didn't go into any detail what to do with them. Waves doesn't store those files as a simple .img or .iso file but instead as .wfi. So even though I had the original firmware always right in front of me, I always assumed it was a proprietary file format that only Waves knows how to decompress. Messing with those files was a gamble to me. It could be really easy, or it could become a week long exercise in forensics that just ends in frustration (And at that point I was already way down the rabbit hole of "compiling a linux kernel from scratch"). So it wasn't until recently that I was motivated to play around with those .wfi files. I started at "Let's open it with 7-zip, just for fun, as if that's going to work, lol" and then it actually worked just like that xD

Nice that you admit all of that should be ignored, to be honest personally I think you should delete your previous posts as they don't help this thread in any way. People that come in here looking for help, directions on how to build the server will just get really confused, there's no need for that.

I agree that my previous posts have caused lots of confusion. I also got quite a couple of emails of people asking for help.
I don't want to just delete all of it, but as soon as I/we have figured out the syslinux part, I'll think about ways to streamline the information to make it more accessible.
Some people that reached out to me via email also reported, that they had issues joining this forum. So I started a topic on gearslutz as well, where I already tried to compile everything I've learned at the top https://www.gearslutz.com/board/music-computers/1311451-diy-waves-soundgrid-server-2020-edition.html

To handle Dynamics and EQs no one needs a server...
Native runs great for that and without latency problems.
There's no point in building a soundgrid server that has no relevant processing power

I don't agree but that's your opinion and I'm okay with that.
I am absolutely not recommending that people should buy a cheap celeron if they set out to build their own server!
My goal was to get a system together, that was built around the same motherboard Waves uses. Fortunately I succeeded, but I had no guarantee it would work. As mentioned before, going for an i3 would have added another 100€ to what was already 300€ in parts.

As for performance, I tried uploading a screenshot, but GDY complains about failed security checks. So here's an example of what I was able to safely throw at the G4930 in Soundgrid Studio:
64x RComp
8x H-Delay
8x API 550A
8x F6 RTA
8x Cobalt Saphira
8x PuigTec EQP1a
8x C4
4x Scheps 73

 

[Whoops]

Thank you so much for explaining krabbencutter,
I understand it much better now
Thanks

As for performance, I tried uploading a screenshot, but GDY complains about failed security checks. So here's an example of what I was able to safely throw at the G4930 in Soundgrid Studio:
64x RComp
8x H-Delay
8x API 550A
8x F6 RTA
8x Cobalt Saphira
8x PuigTec EQP1a
8x C4
4x Scheps 73

Thats a lot, it's not only EQ and simple Dynamics.
I would say thats quite a nice that it run all of that special the 8x H Delays, and The 8x F6 and 8xC4

Nice.
Congrats for your build

I hope one day I can achieve what you already achieved and build my server also
Regards

 

[joaquins]

Thanks for all the info krabbencutter!!! I'll get into it.

...
- if you're on Windows navigate to C:\ProgramData\Waves Audio\SoundGrid Firmware\SGS
- [don't have a mac, so someone can maybe share the folder path here  )

Done in MAC, the dress is pretty much the equivalent one.
[⁩HDD] ▸ ⁨Librería⁩ ▸ ⁨Application Support⁩ ▸ ⁨Waves⁩ ▸ ⁨SoundGrid Firmware⁩ ▸ ⁨SGS⁩

- use 7-zip (https://www.7-zip.org/) to simply extract the .wfi file to your bootable USB stick
- [still don't have a mac  appreciate if someone can try this on mac]

Keka is the 7-zip for Mac, https://www.keka.io/es/
It doesn't recognize the .wti and it runs a compression on it. Changed the extension to .zip and seems to work fine, the decompressed folder has a /boot, /EFI and /version folders in it and a bunch of stuff in them.

I still haven't tried it but maybe later tonight I'll keep at it, I need to get a syslinux running and then drop it in, see how that goes.

Now you have it working, do you think there's a way to use an audio output from that server? That could potentially work as a soundgrid interface, they shouldn't be much more than a server with an audio interface running on them... There are also de IO .wfi on the next folder, (... ▸ SoundGrid Firmware⁩ ▸ ⁨IO), so there should be something in there to pull that thru.

JS

 

[cyrano]

That's about as far as I got on a VM. So I'm following with increasing excitation... 

You might also want to try "The Unarchiver" from MacPaw for compression needs. Keka is new to me. I'll be trying it later.

See:

https://macpaw.com/the-unarchiver

MacPaw has had a bit of a rocky security history, making other stuff that's next to adware. They've been around for ages, though.

I've been using The Unarchiver for many years and never caught it accessing the network when it wouldn't need to. And it doesn't connect at all if you switch off auto-update. So this one is OK, but I stay away from their other products in general. "Clean My Mac", for instance.

 

[bernatvm]

Once you have the Efi bootable usb, the .wfi file is a fully working OS or you have to install a linux distro and then add the contents of the wfi?

EDIT: Is the fully working OS. To create the UEFI USB on OSX I used this guide

https://www.pyrosoft.co.uk/blog/2013/01/09/creating-a-bootable-usb-stick-from-osx/

Ignore 3rd step which is installing a Linux OS you should find on the .wfi file as explained before.

 

[macpherson]

Got it running on an Intel NUC.!!

I had previously partitioned and made the USB stick bootable by using Console on a PC. This might not be necessary but I downloaded a version of linux to the stick to see if it would boot. It did.

I then deleted all the files pertaining to that operating system and Expanded the file from the waves installer.
I was delighted when four little penguins appeared at the top of the screen and loads of code zoomed past.

It then got stuck in the no network loop that krabbencutter mentioned

Once I had it attached to the network it fired up and was seen by all the Waves applications.
In SG Studio it appeared as a sound grid server running in "safe mode" whatever that is.

I googled how to firmware upgrade, you need to hold down shift F and click the little window in the bottom right of the server window. This allows you to reinstall the firmware legitimately after I done this the server wasn't in safe mode anymore. I assume this does a factory install and tidied everything up. But be careful because you can choose where the software goes and it's very easy to accidentally install the firmware onto the main hard drive of the PC rather than the USB stick! As an aside if I boot up without the USB stick Windows comes up as normal and appears to be unaffected.

Well done everybody. Looks like we've got there! Many thanks.

 

[joaquins]

Well, here I am again...

Got it to boot! that's a good one. It doesn't show in the waves applications thou, maybe it doesn'y like the 100Mbps network, maybe the network controller, I have another mobo with intel lan controller, could try that, could try connecting directly the two PCs to see if with gigabit network it likes it better.

Is there a way to set the IP on the server? there's something about that in the lines below the penguins.

JS

 

[macpherson]

Well, here I am again...

Got it to boot! that's a good one. It doesn't show in the waves applications thou, maybe it doesn'y like the 100Mbps network, maybe the network controller, I have another mobo with intel lan controller, could try that, could try connecting directly the two PCs to see if with gigabit network it likes it better.

Is there a way to set the IP on the server? there's something about that in the lines below the penguins.

JS

Got to be Gigabit I think.

If you have got it to boot I think you're very nearly there, I don't think it will boot up if it can't make the network adapter work. The IP stuff sorts itself out.

 

[joaquins]

Got to be Gigabit I think.

If you have got it to boot I think you're very nearly there, I don't think it will boot up if it can't make the network adapter work. The IP stuff sorts itself out.

Thanks macpherson, I'd just tried with a crossover cable but it didn't work, the pc didn't even recognize as a wire was connected, but the wire looks ok on the tester.

I'll go and get a gigabit switch and try again, seems like all the stuff I got around is old 100Mbps, time to upgrade, should be useful anyway.

I'll be back later with some, hopefully, good news.

The last thing I get in the server is press enter to activate this console. I tried a ping connected to the 100Mbps network and get networkis unreachable  as response, lan connector lights stay off, same at the router, for me that looks like it's not recognizing the lan controller. If it still doesn't work with gigabit I'll try with the other motherboard with intel lan controller.

JS

 

[joaquins]

It's working!

Swapped the motherboard for the one with the Intel LAN Controller, gigabit switch, and voila! Didn't tried with the 100Mbps yet, just to know, and a few things to polish but the server is actually working. With the new mobo it did looped when lan wasn't connected, didn't do that with the other one which has a Realtek LAN controller, just jumped right into the end, and behaved as the lan controller was turned off.

Thanks all for the support.

For the record, I'm running a core i7 4770K, 16GB of ram, on a Gigabyte Z87X UD4H. Haven't done any performance test so far. I haven't checked how to monitor the usage but I don't think I'll be running out of ram any time soon, just a matter of how far would the old i7 go.

JS

 

[GreyboxAudio]

Hey everyone,

glad to see some of you already got it working
One of the posts got me thinking and I'm happy to report I found an easy solution to burn the Bootloader on Windows!

Download Rufus from here https://rufus.ie/
Once open, toggle "Show advanced drive properties" (below the "Partition scheme"-Dropdown).
You don't need to click anything here, instead click on the "Boot selection"-Dropdown.
Now you'll be able to select "Syslinux 6.04".
That's it. Make sure you've selected the correct USB drive and hit START.
You'll get some warning dialogs, as well as some dialogs where you need to allow Rufus to download syslinux.
Once everything's complete, extract the .wfi file and boot up your server  8)

https://github.com/pbatard/rufus/wiki/FAQ#list-of-rufus-advanced-features-and-cheat-modes

 

[GreyboxAudio]

It's working!

Swapped the motherboard for the one with the Intel LAN Controller, gigabit switch, and voila! Didn't tried with the 100Mbps yet, just to know, and a few things to polish but the server is actually working. With the new mobo it did looped when lan wasn't connected, didn't do that with the other one which has a Realtek LAN controller, just jumped right into the end, and behaved as the lan controller was turned off.

Glad to hear! As macpherson mentioned correctly, Soundgrid has to be Gigabit:

SoundGrid is an Audio-over-Ethernet technology developed by Waves. It allows to stream high counts of digital audio channels (up to 128) and to process that audio through plugins at ultra-low-latency (as low as 0.8 ms). All taking place over a 1Gb network.

https://www.waves.com/soundgrid-101-part-1-what-is-soundgrid

For the record, I'm running a core i7 4770K, 16GB of ram, on a Gigabyte Z87X UD4H. Haven't done any performance test so far. I haven't checked how to monitor the usage but I don't think I'll be running out of ram any time soon, just a matter of how far would the old i7 go.

You can see the CPU usage in your Soundgrid Host application (Soundgrid Studio/SuperRack/LV1) and to check the memory enter the console on your server and type in cat /proc/meminfo .
At the top of the output you'll find MemTotal which is the amount of memory installed and MemFree which is the amount of available memory. The output values are in kilobyte.

 

[bernatvm]

Succesfully installed on a very old i7-870 with 4gb of ram. It doesn't run a lot of plugins but for the fun of playing, It works and its detected properly.

I think the network card on that machine is a realtek though, I have to double check that.

Any ideas about ryzen compatibility? Any ideas on how to estimate performance before building the system?

 

[GreyboxAudio]

Any ideas about ryzen compatibility? Any ideas on how to estimate performance before building the system?

On my MSI B450m Mortar Max & Ryzen 3 1200 it boots, but doesn't connect. ifconfig shows a lot of dropped packets.
Soundgrid OS is specifically tailored towards Intel CPUs, so even if it would run on a Ryzen system, there's probably a lot that could go wrong.

To estimate performance, you can check the Waves website. They publish benchmarks for each server, which you can find on the respective product page in the 'Tech Specs' tab.
https://www.waves.com/hardware/soundgrid-servers
Unfortunately the benchmarks aren't consistent across server generations. The numbers for H-Reverb are a good measure imo.
If you can't find the exact CPU model for a server, check the A&E specs PDF at the bottom of the product page.

 

[joaquins]

Glad to hear! As macpherson mentioned correctly, Soundgrid has to be Gigabit:
https://www.waves.com/soundgrid-101-part-1-what-is-soundgrid
You can see the CPU usage in your Soundgrid Host application (Soundgrid Studio/SuperRack/LV1) and to check the memory enter the console on your server and type in cat /proc/meminfo .
At the top of the output you'll find MemTotal which is the amount of memory installed and MemFree which is the amount of available memory. The output values are in kilobyte.

Thanks, it did pick up that the server was in the network, wouldn't let me use it... Just to check it is useful to know, I guess.

I didn't have a lot of time to play with it but I did loaded a bunch of channels in the soundgrid rack witha an H-delay, an abbey road room, and a few other stuff and then a couple more on the daw, mainly Abby road rooms, till I got it to 96% CPU or so, memory was mostly free, I was only actually using the few in the daw, i don't know how it handles channels without signal. Didn't drop a beat, pretty sweet to be able to extend the processing power. It probably will come useful in the studio, as at home I have a pretty powerful desktop for my daily use, not so much the one in the studio. I plan to edit video at home and audio in the studio so there's where I might find it useful.

For the Ryzen, I still think it's likely that the problem are the network drivers, it should be possible to add them to the system, as I had the other motherboard lying around I didn't bother to look at that option, I didn't even expend time checking what was in the bootable, didn't even check what's there after the first boot.

My only bummer now is that the mobo with the intel lan doesn't have M.2 for the SSD I have for it, the other one did, but I think I can swap the mobo for a third one I have which has in a working PC wich has both features, which I don't need in it, as is the home general purpose PC.

Great community here, I'd love to hang out more often, as I used to, 32h/day would be nice I guess.

JS

 

[macpherson]

$s per C6

Hi now we have it working would it be possible to work out a sort of bang per buck table. Looking on eBay there are plenty of servers for sale. Many are under the hundred pound mark in an ITX(? small) size case and might work well if they use the onboard intel NIC. I don't think lots people would find the full depth 1U servers practical.
My 2015 1.6Ghz NUC works but maxes out too easily. Powerful new NUCs are too expensive (for me).

The C6 has generally been a good measure as it has been a fairly power hungry plug, perhaps people could list what they are running, how much they picked it up for, and how many C6s it runs in their DAW before maxing out!

Just a thought, hope its not too off topic.

Or if someone spots a common computer that would make a good candidate they could mention it here.