Any IT people here? Stacked switch vs daisy-chaining-what do I need?

[Brian Roth]

Hi Bri,
The construction was done years ago. All the rooms already have cat 5 wiring, jacks, etc. 

Unfortunately when the old switches were taken out they literally cut the Ethernet cables and so indeed a lot of the labeling for those cables is gone. I got a sniffer and tester to start to remap everything...good times!

Wow....bummer having the cables chopped apart in the 'switch room".  Means that after you "sniff" out each run (and add new labels) you have to punch them down onto a patchbay.  Be sure to have correct tools and follow the rules (like no more than 1/2" to 1" of untwists going into the patchbay punch downs.

Speaking of which....as you tone-out each of the wall jacks, spend a minute/three on each jack...unscrew the plate from the wall to be sure there is no termination "BS" in the wiring.  I've seen some stupid errors/kludges behind the wall plate.

Bri

 

[Mbira]

https://www.studytonight.com/computer-networks/osi-model-datalink-layer

https://www.petri.com/csc_routers_switches_and_firewalls

Thanks boji!  Once I get things up and running then it will be a lot easuer for me to convince the others to put money into other things.  Can you recommend what sort of routers I should be looking at?  Cheers! 

 

[Mbira]

Wow....bummer having the cables chopped apart in the 'switch room".  Means that after you "sniff" out each run (and add new labels) you have to punch them down onto a patchbay.  Be sure to have correct tools and follow the rules (like no more than 1/2" to 1" of untwists going into the patchbay punch downs.

Speaking of which....as you tone-out each of the wall jacks, spend a minute/three on each jack...unscrew the plate from the wall to be sure there is no termination "BS" in the wiring.  I've seen some stupid errors/kludges behind the wall plate.

Bri

Thank you!  I will definitely listen to your advice!  Thankfully the first room - with the fiber drop - still has all the cables connected to the patch panels, so at least I can get that wing up and running before I move on to the next two wings. 

 

[boji]

what sort of routers I should be looking at?

If you don't want to learn vlan and security, the wizards on the edgerouter (link below) will give you roughly 10 subnets with minimal config. Your switches should be able to distribute the multiple ranges but there will be no security if someone accesses the ports or hacks a machine. This sounds like an acceptable risk since they are not paying you for net sec.
https://www.amazon.com/Ubiquiti-Networks-EdgeRouter-Passthrough-ER-12/dp/B07MX6H3T4
Note the vid I posted earlier, you can do vlans on the edgerouter but it takes a little more work than pfSense.

If you want to learn vlans, but don't want to load the OS on a fresh machine, the netgate (not netgear) boxes running pfSense can be configured with additional security in mind. There are other brands, other software like untangle, sonicwall, sophos, but most are subscription based.  When creating virtual lans in the firewall/router box, you'll need access to the managed switch's webpages to assign complimentary vlan id's to ports.

As for Wifi, don't buy wifi routers, you want access points spread out over the building.  Earlier link to the Unifi AP 3-pack would take most of the work out of setting up a mesh and guest access.  Plenty of tutorials on youtube for setup, which is easy.

 

[Brian Roth]

I'll soon quit harping about the wiring plant, but definitely study what you have, since you've inherited "some else's mess"  lol.

Determine if the jack connections are T-568B (most likely for a newer installation) vs T-568A and stick with whichever protocol is used, end to end, and throughout the entire wiring plant.  Newer switches seem not to care, but.......<g>

https://incentre.net/ethernet-cable-color-coding-diagram/

Look for "stupid pet tricks" hiding in the walls/ceilings.  I've seen "bootleg stealing" of the blue wire pair in a run to connect a POTS line (for a fax machine) in the same wall plate....totally screwed up the ethernet connection on that same cable.  It kinda worked, but was sluggish and unreliable in a 100 BaseT system.

I have other wiring horror stories....but I'll stop.  lol

Just sayin'....be aware when dealing with something you had no control over!

Bri

 

[Mbira]

boji-as I learn more, I will definitely be wanting to create vlans for each room.  There are a few businesses here that use a few rooms, so I will set up vlans for each business.  This will be good job security for me. ;-) 

 

[PRR]

> the owner is currently paying $1500 a month for this internet service
> three Poe HPE 48 port switches
> a facility with about 75 rooms
> each business (band, pr, etc) has a single room, so one business will only have access to 1 or two ports

This is looking like a BIG job. Not a spare-time job, at least at first.

The fiber costs $18k/year. Switches last 5 years so your $350 investment costs-out at $70/year, 0.4% of the elephant in the building. This may be a Good Deal; they are good switches and you don't need latest/greatest, and those who do often tag-sale their old boxen cheap. But you can't penny-pinch your way to complete happiness justifying a $18k/yr ongoing cost.

You have more ports than I did when I was Systems Administrator. And fiscally I only had "one client" (3 departments in one university); you have ~~70 clients who may not all play nice together. (The dept Chairs or Dean could sort-out disputes better than a Landlord who needed rent-money.)

You can't run a "business" on 1-2 ports. Yes, some of these bands are not "real businesses" like a doctor or an accounting firm. But 4 guys in a band, a lover for each, plus a couple printers, is peak 10 devices.

However Wi-Fi rules today, so 2 ports per room is maybe two more than will be used. (Kids don't know what an ethernet hole is.) Wi-Fi will be *demanded*. It would be absurd to have 25 "per tenant" Wi-Fi boxes in one wing (but this WILL happen if the landlord doesn't do it). There are mass-WiFi boxes for schools and motels; this is not something I have done. I could physically locate a wall-port and deal with an offender; a Wi-Fi bad apple is harder to track.

Your short-term job needs much more wire-work than I ever did (installation was done "by others").

You will have bad apples. High-traffic servers inside the net. Machines infested with viruses. Much of my hair was lost when network central reported such problems to me and I had to track-down the offender and fix the problem. You don't even have any such monitoring (excess or suspicious traffic).

You clearly have only slight background in IT management. That's OK, many of us were thrown in the fire under-prepared.

My job grew from fixing audio to fixing a dozen+ isolated PCs to managing a hundred ports. I looked at what I was doing against what I was paid for my legacy duties and fought hard for a $13k/year raise in pay and title (even the whoremasters in Personnel agreed this was justified). While I know _you_ will do it "for art", the time and learning-curve distracts from doing your own art. *Fight for fair money!* This may be money for you to do drudge-work and panic-response, or money to hire an IT person, and of course money for *good* gear that does not fight you or fall out of the ceiling. $11.11/port is absurd. I was allowing $100/port/year for maintaining things, between my pay and incidental upgrades. Long-term upgrades would double that (but such projects were above my pay-grade and annual budget).

 

[Mbira]

PRR: I am offsetting my work with free rent-which for now works just fine for me.  If I have to renegotiate in the future because of time, etc, that's something  can certainly do. 

I'm doing what I can with the budget that is available to me.  Indeed the vast majority of people are not even using the ethernet.  The primary reason I'm doing this is because my studio is on the far end of one of these switches and I use a NAS for offsite backups.  *I* want wired internet.  I think there are currently maybe 10 other people using wired internet in the whole place (based on the number of random switches and router currently cluttering that room.  Everyone else is pulling off the wifi.  I think I'll be doing something like setting up wifi access in each of the three wings and then keeping each of those on their own vlan with a bandwidth restriction.  That will probably take care of 90% of the internet usage in the building.  Then the rest will be dedicated vlan ports from the switches to those of us that are actually wanting the hardwired access. 

Am I not able to monitor specific vlan usage from a managed switch?  I thought that was one of the features of those....it's true I am very new to this!

 

[boji]

Am I not able to monitor specific vlan usage from a managed switch?

Get into the OSI model, it will explain the reason why not.  Switches work in layers 1 & 2, and deal with data like streets and traffic lights manage traffic. You wouldn't ask an exit sign to explain to you why a driver is on the road or how many cars have taken a particular offramp.

When you monitor for 'usage', you're above layer 2.  Routers work mostly at layer 3. Firewalls are at layer 3 & 4, and send those layers of information up to the software layers (5,6 & 7) to be interpreted and monitored. 

Edit: A "managed switch" means it is a switch that can be managed by the right kind of packet headers/wrappers (sent by a managing device), not that the switch itself manages the network. 

I  didn't figure in this distinction until my 'trial by fire'.  Looks like you are certainly heading towards yours. 

This is a nice pdf that shows the extra header information contained in layer 3:
https://nsrc.org/workshops/2017/caren-cndo/networking/cndo/en/presentations/Layers.pdf

Some unsolicited advice (tipping the hat to PRR and Mr. Roth):

• Don't talk to anyone around your shop about what you want to do/add until it's already been running in the office for some time.
• 
  Read some tuts, watch some vids, buy some equipment and Lab this stuff at home, so when it's time to implement, you already got most of it figured out.
• 
  After the business networks are up and running, convert the home lab into a sandbox. No one at the office needs to know you returned an item because it melted your brain trying to integrate it into the network, or that firmware updates on some devices breaks the business network.
• 
  If you really want to get confident, have your friend or significant other occasionally throw a monkey wrench into your home lab to convert some of your troubleshooting into kb/muscle memory.
• Buy a three ring binder and start a kb so the people that come after you won't hate you.

 

[Mbira]

Thank you guys for all the help and the advice. 

Boji, that last PDF helped me understand more about the layers. 

It looks like I will be able to get the Ubiquiti edgerouter that you mentioned after talking to the folks here. 

Some basic router questions:

With the router does that mean that I will be able to monitor and control the amount of date that individual vlans are using?  (I guess that is handled on "level 3"?)

Is there any benefit or difference in performance to putting wireless hotspots directly connected to the router ports vs in the switch ports?  We will be setting up at least three of those-probably closer to 6.

In general, is there anything I should (or should not) be plugging in to the Router LAN ports other than the managed switches? 

 

[boji]

will be able to monitor and control the amount of date that individual vlans

Edgerouter will let you create seperate networks real easy, and see traffic as timeline graphs, But wrt QoS / band limiting, you will likely have to add some config lines. They call it a Host Fairness Queue (HFQ).
https://community.ui.com/questions/Per-IP-Bandwidth-Limit/b0e166f7-ba01-4db9-9fd5-05576728d4f3

In my experience, pfSense requires learning how to set GW, LAN, and NAT's and point subnets to physical ports, so it will initially require more time to learn than the Edgerouter's wizards, but once you get into bandwidth limiting, pF will become easier to configure than edge, which is not as full-featured or as powerful. Granted the Netgate box is more expensive, but pfSense has way more features than the edge router and a better firewall rule system.  Plenty of tuts online for pfSense.  But either one will get the job done.

Is there any benefit or difference in performance to putting wireless hotspots directly connected to the router ports vs in the switch ports?

No, not really. Normal to run them off the PoE switches. They want PoE power anyway. The unifi controller software makes adding AP's a breeze. You'll set wifi speed limiting in there too. Btw, If you had Unifi switches you'd see that same control software that is configuring AP's would also find the switches, and let you assign vlan tags. If you added a USG, you'd be creating your subnets there too (but band-limiting the hardline network on a USG is still command line). 

In general, is there anything I should (or should not) be plugging in to the Router LAN ports other than the managed switches?

Well yeah keep it simple. Gateway into router's wan port, router eth ports to switches. New stuff that wants ip's goes into switches.

 

[Mbira]

Progress is being made.  I have the switches wired up and got the Ubiquiti Edgerouter 12 up and running today.  I have everything set up all just on the same subnet and now I'm trying to figure out how to set up the VLANs.  I figured out how to get a different subnet for each physical LAN out of the router, but I'm trying to figure out how to create Trunks in the EdgeOS. 

I just started on this part today and pretty overwhelmed-not quite sure how (and why) the Edge software is so seeming needlessly complicated in setting up these VLANs....  I can't say I have any questions yet...still just trying to wrap my head around the software enough to have anything intelligent to ask. 

 

[boji]

Hack away Mbira!      Glad you've got the GW talking the the Edge.

Have you copied the vlan id # you're assigning in the edgerouter also to a specific port on one of the switches?

After that, try to ping the router from a laptop that you've given a static ip that is on the same scope as the vlan, plugged into a port on the switch that is tagged to pass data across your vlan id.  Make sure the laptop's ip is not a dupe of anything else plugged into the network.

Here's an example vlan setup on Edgerouter:
https://www.youtube.com/watch?v=grg8TxsSops

Edit: He has the benefit of using same brand router and switch , so control software/submenu will look the same.
Setting requirements/fields should be nearly the same for your switch, just a bit harder to locate?

Also take note-- he's adding a vlan under a /16, so second octet is acting like the traditional third octet on a /24!  (here lies a method to distribute all your room subnets down the road...class B, yer savior)

 

[Mbira]

Reading over the HPE switch manual, I think my issue has something to do with setting the tagging.  Are these assumptions correct?

I want to create three "Trunks" that will be tagged and those three trunks are the three LAN ports that I will use to come out of the Edgerouter in to each switch. 

On each individual switch I want to set the port for those trunks to be trunks and tagged and I want every other port to be untagged (I am assuming that every port will only belong to a single VLAN).

I hope my question makes sense.  My primary question is whether these three cables that are leaving the router and going in to each switch are what are called Trunks? 

Regarding the 20+ VLANs on each individual switch....I am definitely conflating VLANs with subnets there.  The vast majority of those really only need to be individual subnets (Though I don't jet know how to make this distinction within the edge or the switch software). 

 

[boji]

issue has something to do with setting the tagging.  Are these assumptions correct?

That was my prediction since you mentioned subnet creation on edge, but did not mention setting switch port vlan id's.  Did you watch the video?

and I want every other port to be untagged (I am assuming that every port will only belong to a single VLAN).

With an edge switch, you configure it to know that a vlan can pass packets over the specified untagged port.  Honestly this is a little confusing for me as well, as I've had some switch brands require setting tags and since I normally setup class C, I set them to 'Accept all'.

Back to the video, three things are essential: Router vlan id's, matching switch port id's, and setup of dhcp clients in the edge that will hand out ip's over ports for all the businesses. But before you go making a ton of id's and dhcp ranges try to get one port to work.  Have you assigned a dhcp range for a vlan yet?

these three [uplink/downlink] cables...are called Trunks?

I only call a cat cable a trunk when it is carrying more than one subnet.  The tree analogy applies. You got branches (subnets) and a trunk to deliver those subs.

 

[Mbira]

Update-all right!  I got the three switches set up.  Today I finally figured out how to get the VLANS set up correctly and got my ports set up as it's own private lan.  Yea!

 

[boji]

Right on! Congrats.

 

[kazper]

The more I understand on the overall system and clients, segmentation and locked down appliances is a must. Look into security software and or appliances at least for your equipment and DATA.

I didn't pull up the specifications for the HP switches, but the pic's shows that you can use SFP ports. You could link the buildings directly to the main switch in a stacked mode for about $300 a drop with custom made cables and closet connectors/jacks. The SFP's cost vary but the generics have worked well in most the stuff I've worked on.

Cheers and Happy New Year!