DIY compatible Waves Soundgrid Server

[GreyboxAudio]

That's insane, I commend you for doing that, I'm sure you're real proud... I would if I was able to, I made it real far, but had to admit that it was easyer to just take it where Waves so fulishly planted it...

I'm using a stock firmware, usb created from the V9.7 image inside the SG software and Whoops's image too, upgrade to V11 worked perfectly...

Thanks. It was just a short lived success though, because it's still not working with my motherboard. And most of the actual Soundgrid-Stuff is happening in the initrd anyway. But knowing how to recompile the kernel, should make it easier for other people to add additional drivers & stuff.

 

[GreyboxAudio]

So about that kernel ...

Waves soundgrid servers run on kernel 4.1.33-rt37. So that's Kernel 4.1.33 with the corresponding realtime-patch.
https://cdn.kernel.org/pub/linux/kernel/v4.x/ (there's a lot of kernel on that page, so use the search function of your browser and download the .gz compressed file)
https://cdn.kernel.org/pub/linux/kernel/projects/rt/4.1/older/ (same as before)

Once download unpack the kernel, copy the realtime patch into the extracted linux-4.1.33 folder and apply it

cd Downloads
tar -xvf linux-4.1.33.tar.gz
cp patch-4.1.33-rt37.tar.gz linux-4.1.33
cd linux-4.1.33
zcat patch-4.1.33-rt37.patch.gz | patch -p1

Copy the kernel modules from the 2016 soundgrid release (sglinux-9.7.99.105) into your source tree.
Just do it manually and replace the files with the ones from Waves.
The files from sglinux-9.7.99.105/kernel_module/wigb/ go to linux-4.1.33/drivers/net/ethernet/intel/igb/
Netx rename the wr8169.c found in sglinux-9.7.99.105/kernel_module/wr8169/to r8169.c and copy it to linux-4.1.33/drivers/net/ethernet/realtek/

sidenote: as of now I still don't have any idea what to do with the /wevent files :

We're also going to need firmware files for the Realtek NICs and while we're at it we can install some additional dependencies needed for the kernel compilation

sudo apt-get install linux-firmware-nonfree libncurses5-dev libncursesw5-dev

Copy the folder /lib/firmware/rtl_nic (going from the root level of your file system) into linux-4.1.33/firmware/

Last but not least, we need a .config file. I've extracted mine from a running WSG server:
- insert a fat32 formatted usb stick to your server and enter the console
- create a directory to mount your usb stick

mkdir /mnt/usb

- mount your usb stick

mount -t vfat /dev/sdb1 /mnt/usb

- copy the config

cp /proc/config.gz /mnt/usb

- unmount your usb drive

umount /mnt/usb

Back in ubuntu-land extract the config.gz file to your hard drive, copy the config file into /linux-4.1.33 and rename it .config
(if it suddenly vanishes press ctrl+h to show hidden files)

Now we're ready to compile the kernel  8) Fortunately for us the Waves Kernel is pretty slim and it should compile in just a couple of minutes.

cd linux-4.1.33
make -j $(nproc)

-j $(nproc) tells your pc to use all available cores for the compilation.

Once that's done you'll find the kernel image named bzimage in the linux-4.1.33/arch/x86/boot folder.

 

[GreyboxAudio]

Now we need to create a bootable usb stick. Remember syslinux, which we compiled in part 1? https://groupdiy.com/index.php?topic=71385.msg941681#msg941681

Carefully follow steps 2-4
https://wiki.syslinux.org/wiki/index.php?title=HowTos#How_to_Create_a_Bootable_USB:_For_Linux

Now create a folder named boot at the top level of your thumb drive and in there a folder called syslinux.
Move the ldlinux.sys into boot/syslinux. Copy your kernel (bzimage) into boot and rename it to vmlinuz
Next copy the initrd & syslinux.cfg found on your WSG-Stick over to your custom stick. initrd goes into boot and syslinux.cfg goes into boot/syslinux.

Your custom soundgrid thumbdrive is now ready for action.

 

[Rasta3]

Hi everyone, just my personal experience.
I gave a try to the image Whoops sent me (thanks Whoops).
I wrote the image on a bootable usb drive and tried to boot a few different pcs from the stick.
The first notebook booted up but I didn’t manage to connect to the server from soundgrid studio.
The second one, a desktop, didn’t boot even if it was an i7 with gigabit intel network card. I don’t know if it has anything to do with the efi/bios options. I’ll give another try.
Lastly the third pc, my girlfriend’s notebook, which did boot from the stick, but no way to connect to the server from the soundgrid studio.
The surprise I had with this third pc was when rebooted, the SSD was completely messed up. Lost every partition and all the data we had on it. Apparently th USB drive wrote an /ext4 partition on the SSD, with no reason and without asking anything. Yesterday I tried to recover the SSD partitions all day long, with no luck. Don’t know exactly what happened but be carefull with this image. I’m not saying it is infected but probably does some kind of formatting at every startup. Just saying do not try it with a working pc, you could have some disappointing surprise.
I’ll pass my day installing win10 tomorrow, then when I’ll find a donour pc for the test with the image will try again.
Are you sure the image is only 16/20Mb?
Is there anything special to do, to have the system up and running?
Thanks

 

[GreyboxAudio]

Oh god, sorry to hear that. But yeah, DO NOT use this image on any hardware before you have physically disconnected all internal storage.
Just to get this out of the way: I'll try to explain why this happened but it is absolutely not my intention to sound like I'm blaming you in any way. It's just a big pile of sh*t happens on your end and I can completely relate.

The soundgrid image is constructed for a single purpose in a specific hardware environment. That's also why it is so small.
But that means also any safety measures you're accustomed to from end-user operating systems are non-existent.
With a real soundgrid server the usb drive is even mounted inside the case and not readily accessible for the end-user.
When you boot the image up for the first time it creates a second partition and decompresses all its contents. In a soundgrid server the USB drive is the only available storage so everything is written back to the usb drive.
But when you have more than one drive connected it is absolutely possible that the OS is actually written to the other drive instead. But to be technically clear: this is not a flaw of the soundgrid OS. Waves have manufactured it to be put in a pre-configured system and shipped to the customer. That we as a community start tinkering with it on our own hardware is completely out of any scope Waves have set for this.

As to the hardware side of things, I've given up for now. There's more to it, than just having the correct Intel chipset and a supported NIC, but I haven't figured out what exactly. So when you boot it up and get an endless loop of
EXT4-fs (sda2): mounted filesystem with ordered data mode. Opts:
pull the plug and throw it out of the window ;D

 

[Whoops]

Lastly the third pc, my girlfriend’s notebook, which did boot from the stick, but no way to connect to the server from the soundgrid studio.
The surprise I had with this third pc was when rebooted, the SSD was completely messed up.\

Why would you use the image on a working PC it's beyond my reasoning.
The image is for a server, a dedicated unit. You need a dedicated machine for that.

 

[Rasta3]

Oh god, sorry to hear that. But yeah, DO NOT use this image on any hardware before you have physically disconnected all internal storage.
Just to get this out of the way: I'll try to explain why this happened but it is absolutely not my intention to sound like I'm blaming you in any way. It's just a big pile of sh*t happens on your end and I can completely relate.

The soundgrid image is constructed for a single purpose in a specific hardware environment. That's also why it is so small.
But that means also any safety measures you're accustomed to from end-user operating systems are non-existent.
With a real soundgrid server the usb drive is even mounted inside the case and not readily accessible for the end-user.
When you boot the image up for the first time it creates a second partition and decompresses all its contents. In a soundgrid server the USB drive is the only available storage so everything is written back to the usb drive.
But when you have more than one drive connected it is absolutely possible that the OS is actually written to the other drive instead. But to be technically clear: this is not a flaw of the soundgrid OS. Waves have manufactured it to be put in a pre-configured system and shipped to the customer. That we as a community start tinkering with it on our own hardware is completely out of any scope Waves have set for this.

As to the hardware side of things, I've given up for now. There's more to it, than just having the correct Intel chipset and a supported NIC, but I haven't figured out what exactly. So when you boot it up and get an endless loop of
EXT4-fs (sda2): mounted filesystem with ordered data mode. Opts:
pull the plug and throw it out of the window ;D

Thanks krabbencutter for the response. I know for what the waves server is intended for so I can perfect imagine it wasn’t designed to be used on a regular machine.
And to answer Whoops I know I need a dedicated machine, I was just trying to sort out why it wouldn’t boot from my desktop pc, the only one which had the specs to give the server a try.
About that, still trying to boot from there with no luck, simply doesn’t boot from that image. I tried different usb sticks, same result. Different usb ports (usb 2 and usb 3), same result. Different images on same usb sticks boot with no problem. I tried a lot of suggested bios settings too with same results.
I hope noone got offended from my post. I’m just trying to alert others about possible risks.
After all, from what I read till now I was certain it was a live usb. Booting from the usb supposedly didn’t touch the internal storage...well now I know that it could have some surprise.
By the way, my girlfriend already has her pc up and running. She should thank me for the fresh install...or maybe not!?!?

 

[maniman]

Hello guys, long time without posting, thanks to all of you for the experiences specially to Whoops and krabbencutter for the extensive info research and genorosity.

For me boots in every Intel chipset I tested, unfortunately the only machine with Intel NIC I own isnt a gigabit one ,is recognized and the link becomes ready and all that but not appears in the inventory.

As we seen in the "Server one inside" youtube video  ...Maybe an "Intel Gigabit CT Desktop NIC" (as the EXPI9301CT mini PCIe)  does the trick ? someone tested  ?

I must assume the server don't care a s*** about the licenses ? the license are managed and would be in the host side all the time ? this is in that way ?

Many thanks, best wishes and health to everybody.
Mani.

 

[Whoops]

I must assume the server don't care a s*** about the licenses ? the license are managed and would be in the host side all the time ? this is in that way ?

Yes, thats correct

 

[Ashdease]

Hi could I possibly get a working copy of the soundgrid server image.  I wanted to try build a server to test out, unfortunately not computer savvy enough get it working from scratch but could try with a working image

I dont have a computer to run this so thinking of building one if I can get hold of the server image,

Spec I'm looking at are

gigabyte h370 hd3 motherboard
I5 9600k cup
16gb ram

Hope this sounds good

Thanks
Ash

 

[Whoops]

Hi could I possibly get a working copy of the soundgrid server image.

If you read this thread you will easily understand how can you get the image

"Usage Rules
READ BEFORE POSTING!"

 

[Ashdease]

My Apologies

If you read this thread you will easily understand how can you get the image

"Usage Rules
READ BEFORE POSTING!"

 

[pillerk]

Hi
Can you link to a complete description of waves soundrid server installation from source (https://www.waves.com/legal/soundgrid-server-open-source-code) ?
What is on the forum here is incomplete and manipulates non-existent folders.

I have been using a soundgrid server for a long time (cca 4 yrs), I also have a GHOST image from it, but I would like to test that if I install it from the source code, it will not perform better.

Thank You

 

[macpherson]

I've been trying to make a bootable Soundgrid Server from the firmware files in the offline installer

SGS_11.2.wfi.wtar.aa  and  SGS_11.2.wfi.wtar.ab.

Are these the correct files to start with?

One of them can be uncompressed to about 16Mb from 6Mb the other kicks up an error.
I've tried putting this on a stick with a boot partition made with Rufus in various configurations and no joy.

Could anyone give  some pointers?


Thanks

 

[Whoops]

I've been trying to make a bootable Soundgrid Server from the firmware files in the offline installer

SGS_11.2.wfi.wtar.aa  and  SGS_11.2.wfi.wtar.ab.

Are these the correct files to start with?

One of them can be uncompressed to about 16Mb from 6Mb the other kicks up an error.
I've tried putting this on a stick with a boot partition made with Rufus in various configurations and no joy.

Could anyone give  some pointers?


Thanks

What I have is SGS_9.7.IMG and it’s 33.6MB

 

[macpherson]

So I've given up on using the SGS11.2 files as I can't find a way of extracting the whole file out of the 2 compressed installer files. Perhaps I can upgrade if I can get the server recognised.

I'm now trying with SGS_9.7.wfi which is certainly the same size as the one Whoops and others are using. The problem I now have is that it's not an ISO or IMG file.  Surely it can't be as simple as changing the file extension and dumping it in a folder called BOOT on the stick? ( I have tried )
I've downloaded a few small Linux images to see what the file structure is and they all have extra stuff like config files and dependancies to make the software boot.

Rufus won't see the file as it stands because it doesn't scan as a true ISO and Etcher has a go but doesn't make anything that works.

Perhaps anyone who has an IMG file could make some suggestions?

Thanks

 

[macpherson]

Ok so having a read through again...

The software unpacks itself on first boot and generates its own partition so I should just be able to use the one file.

 

[Whoops]

Hello macpherson,
thanks for joining.

Unfortunately with the whole Covid thing all my Live Sound work (around 65% to 70% of my annual income) stopped completely and with the financial restraints at the moment and no use for it probably in the next year (Covid is here to stay) I still haven't done my server yet.
So I've just been helping people by sharing the image but I haven't done the server myself neither do I have experience in doing it.

I know some members around here have done it already with success,  it would be great if the people that did it could chime in and give a little bit of help and instructions.
Thank you so much

 

[macpherson]

Thanks for your reply Whoops. I am in sympathy with you. Yes we are in difficult times, I play music and engineer but since moving out of London 15 years ago to a smaller place I've had to turn mostly to other ways of  making an income. I make props for events and parties.... It's all gone very quiet....

But I'd like to figure this out!
Thinking about this last night, (I slept badly), I just can't see the idea that the install file from the offline installer will work directly on the usb.

My hunch is that bits of that file are put in the correct places so that it is seen as a legitimate OS when the server is turned on. Which is why I think that anyone who has the cloned image from a legit server starts with all the parts in the correct place. This is not the case with the firmware file from the installer which will not be seen on start up (but I'd be happy to be proved wrong!)

Scrolling through the software in notepad I noticed that towards the end in amongst the gobbledy-**** there are references to specific computer setups, processors and lan cards so it might be possible to see if your hardware is compatible by scanning this.

I think I really need to get hold of get hold of an image file.

All the best Whoops thanks for your reply, stay well.

 

[joaquins]

  Hi everybody, long time since I last posted something here, nice to get around to come back!

  Well, I was looking for the server after I saw the price and specs and got nice surprise to see a recent topic here about it.

  I'm starting a studio with a friend, casually his computer is an H81M so chipset should be good to go. Of course would be too nice for it to have an intel LAN, it has a realtek gigabit, can't find the exact model now (baaaad internet connection right now) and I have a slightly better computer which could very well end there as I upgraded from it an year ago.

  Other upside, I do have some experience in software, and with krabbencutter's explanation I could very well get that working, but I rather not, nor I have the time... So I guess I'll PM someone with the image, try to make it tick and come back to add some lines to the thread.

  My question to the ones who got to play with it, is it really no way to add a lan driver for whatever chip is in the system?
  I'm thinking manually adding the driver to the usb drive putting it back on a linux machine after the first boot. Or I'm missing something? I guess I'll find out by myself when I get to play with it, unless for some reason it takes my system as is.

Best regards!
JS