Importance of Baselines

one of these things is not like the other
In my world, having a standard baseline across servers is important. For example, I am responsible for maintaining a few hundred servers spread across a geographical area. By having a standard baseline across the servers its easy to maintain updates, troubleshoot issues, and in general keep things running smoothly.

Well, recently we received a hardware refresh consisting of new servers, storage, and networking equipment. While installing the equipment we made a system baseline for our sister sites. The theory is that each site would have the same hardware, the same software, and then if one site has issues or upgrades something all sites apply the change.

This is in stark contrast to my site and other sister sites all being islands of development and maintenance. Lessons learned didn’t get shared and when they did they got morphed into site vs site competition.

So one would think we are forming a standard baseline. Sadly that idea, that concept, that goal has failed so far. The organization responsible for providing the hardware decided to change things between our site and the next. Where we have a Cisco 6500 switch they have a 4500. Where we have a set of blades and another set of standalone servers, they only have the blades. Where our blade chassis have one type of built in switch, they have another brand which supports different features that we are using.

So in review. If your environment looks like this:

  • Multiple subnets/VLANs on each server for traffic segregation and security
  • Bonding/LACP at the server and between switches for redundancy/throughput

And you can’t hire enough smart network engineers that understand trunking/bonding/VLANs then don’t switch your hardware mid-game, and then don’t call the site that got it working asking for help.

Written by ruckc | Tags: , ,

Google’s Batteries

its just cool, eh?
In the recent CNET article Google finally published specifications on their servers. They have batteries mounted on the chassis. Instead of going for giant data center batteries, they went with individual batteries on every server. I would be very interested to see their statistics on lifetime expectancy of the batteries. Plus, their power supplies appear custom manufactured since it provides only 12VDC to the motherboard, and the motherboard handles the voltage stepping. Its anyone’s guess how long it will take manufacturers to start selling server/workstation equipment based on the simplicity of it the design.

Written by ruckc | Tags: , ,

50% blade failure

servers or something close

So at work, we needed to have some work performed on one of the SANs. To do the work we went ahead an powered our blade servers utilizing the SAN off. After the SAN was repaired (through means of a cold boot) 16 of our 32 BL25p blades refused to power on. There was no pattern to it, no single blade enclosure, no unique software. When you pressed the power button on the blades they just refused to turn on. It was more like they thought about turning on for a moment and then replied “Hell No.” Luckily we made due with some spares and reallocated blades from another similar setup elsewhere, but seriously how does this happen?

According to HP this is an issue that we should of fixed with upgrading iLO back in the spring. You gotta love quality engineering.

Written by ruckc | Tags: , , ,

Graph Fun

connections two
I enjoy graphs, not the charts that Excel can generate, but Graph Theory Graphs.  The kind of graphs that you can use to calculate dependency matrices, identify network problems, and design flow charts.  The kind of graphs that can contain millions of vertices and millions of edges. This was made possible by finding JGraphT.  It’s a java graph API that focuses on handling the data structure of graphs.  My API wraps a ListenableGraph and creates cooresponding Components for the vertices and edges.

I bring this up because I may have finally wrote a decent Graph API for Java Swing.  Its not much of an API at the moment, but it does allow positionable labels on edges and draging of vertices around.  The API doesn’t do Canvas drawing like some of my prior attempts, nor does it mix Components with manual painting.  It creates a component for every displayable element in a graph.  Currently I have 3 main components with 3 extended JPanels containers to layer the 3 components, VertexComponent, EdgeComponent, and EdgeLabelComponent.  The VertexComponent ideally will be styleable by storing Objects that implement the Map API as the Edges and Vertices in the data structure.  The EdgeLabelComponent will have a Event/Listener framework for updating it or attribute based updating.  I am currently leaning toward not implementing the attribute based labels since the underlying JLabel’s preferred size will resize as contents change and it would be unwise to do that in the middle of repainting.

If there is any interest in the above API email me, I may actually try to finish it.

Enough Java, time for rant.  Graph theory means alot to me because it has so many ties into our daily lives without knowing about it.  For instance, as a play on XKCD’s 173. Imagine trying to plan for a thanksgiving dinner with 3 immediate family members and 27 additions (spouses, children, cousins, “family” friends). Now trying planning out optimal seating so that everyone has appropriate conversation and all kids are near a responsible adult without seating cousin Eddie next to Mother In-Law due to personality conflicts. These issues can be addressed using a Force-Directed Layout algorithm. This will only work if you can assign numeric values to who needs to sit next to who and who should be clustered, and who should be as far away from each other as possible without being obvious. Graph theory has the ability to solve problems from optimal building plans, any complex scheduling (think college & high school class assignments), and near infinite other problem sets. Nature’s circle of life operates using a dependency graph with built in priorities. Due to the wide reaching aspects of Graph Theory if one can simplify it such that it can help average people solve complicated problems you have a fairly wide reaching business model on the condition you can profit from it.

Written by ruckc | Tags: , , ,

The Beginning

once upon a time - typewriter
This is the beginning of the SQ blog in theory.  The intention is so that I describe my daily distractions from work.  I currently work 7 days a week, 12 hours a day so the distractions are sometimes frequent and many.

Written by ruckc | Tags: ,

WordPress | Aeros | Extplorer