Adding Capacity? Turn the Spigot Slowly.

Success does not consist in never making mistakes but in never making the same one a second time. - George Bernard Shaw

 I've seen this more than once on two different teams where I work:  on a network upgrade and on our BI Tools Team.  Capacity is identified as a problem which is causing incidents.  The solution is to submit a change to add horsepower.  Come Monday morning, new incidents arise.  To the clients the systems still aren't working, but to IS the problem has simply shifted.

When you add capacity to a system, you're fixing a bottleneck but are all of your downstream systems equipped to handle the deluge of new data coming their way?  You could be breaking South Fork Dam over Johnstown.  When you open the floodgates, are your systems ready?

In my case, an upgrade to SSRS server capacity threw more at the report source databases than they could handle.  After a day of intense investigation, we realized what needed to be done.  Adjust queue depths and indexes and upgrade disk frame technology and database server memory and CPU.  Additionally, the application support team took the most important and often overlooked step of report query tuning.  Soon everything was humming like a well oiled machine.

The lesson learned was that bottlenecks are systemic:  they don't go away they just move.  Before adding capacity, if you're lucky enough to be able to replicate Production in your Test environment, test the upgrade thoroughly.  If upgrading Test to match Prod is too expensive or wouldn't yield accurate results due to differences is load, then turn the upgrade spigot slowly.  The temptation to do everything possible all at once can be hard to resist, but changes are best done one at a time so the effects can be observed.  If multiple changes must go in at the same time, use administrative tools to restrict the effects, allowing you to turn the knob slowly and keep an eye on things.

Too Many Choices, Not Enough Metadata

In my efforts to do a bathroom remodel I've realized a common point of frustration: search fatigue. Not to pick on Lowe's in particular but rather case in point because I like them, whether the search is bathtubs, tile or fixtures a customer's ability to filter search results seem limited mostly to the standard three: price, rating and brand. However if there is an online selection of over 200 Delta shower heads, and I'm looking for an antique bronze colored hand-wand-in-shower-head, it is a frustrating endeavor. 

So now some advice to retailers, particularly to home improvement retailers where a bad online search capability can cost you thousands: allow your products to be pivoted to the maximum number of common product dimensions. If your customers are buying windows, they want color, size (if not custom), material*, efficiency*, number of panes, open method, and whether replacement or new construction, in addition to price, rating and brand (*in fairness Lowe's had material and Energy Star certifications as categories- but there are four binary categories for Energy Star on their mobile site which is confusing).  You already have product specs on your site, now unleash the metadata possibilities!

Don't beat yourself up for being an almost lottery winner. There's no such thing.

I was reading through an article with recommendations on how office lottery pools can fairly play the lottery as a group.  There were some good points, but two in particular that got me thinking.  This led me to conclude that two of the points taken together are fallacious.  In a nutshell, one says "let the computer pick the numbers" and the other says "play every week so you don't miss the week that hits".  However, the truth is, if the computer picks the numbers on a week you didn't play, it would not have picked those same numbers if you had played.

Quick Pick lottery numbers (I use the Powerball term to cover all computer generated tickets) are generated at each terminal by a random number algorithm probably seeded by something unpredictable like atmospheric or electrical noise.  Many people have the misconception that if a lottery winner comes from a store in their neighborhood, they were much closer to winning than someone on the other side of the country because if they had been next in line instead of THAT person, they would have won.  Not true.  If we assume that your being there has no impact on the seeding mechanism of Quick Pick, the time factor alone is mind numbing.  There are limitless ways to interact with the checkout clerk to ask for a lottery ticket and if the button that generates that ticket were not pressed at the exact millisecond as before, you may as well have been a million miles away from that store.  Maybe the clerk pauses because you remind him/her of someone, or maybe you preface your request with "Um", or maybe the other winner had asked for tickets three times to be understood because of their accent.  Regardless, different numbers will be generated.

So let's assume that your lucky office pool buys some tickets and wins without your participation.  If you called in sick that day, your team merely thinking about your absence changed their behavior enough to generate the winning circumstances.  Said another way, if someone wins from playing Quick Pick, it's because everyone and everything behaved exactly as it did and the game was played at that exact location at that exact second.  This sounds like science fiction, but it is not.  This indeterminate randomness is absolutely necessary; otherwise, someone would have cracked the lottery by now.

This knowledge may not allay the jealousy you feel from seeing your coworker's new found wealth, but it can at least spare you the misery of continuously reliving the "what if" scenarios in your head.  About the only thing you can say with near certainty in this case is "Damn!  If only I had played the lottery with my team that day!  Then they would not have won and I would not have the misery of closely knowing the jackpot winners and seeing firsthand how much fun they get to have!"

sources:  http://www.molottery.com/powerball/understanding_chances.jsp
              http://www.ehow.com/how_2073998_enter-lottery-coworkers.html

How Apple is Better even When it's Worse

The latest U.S. Cellular commercials are trying hard to obfuscate their lack of an Apple offering by highlighting their Android lines and by poking fun at the Apple faithful as lemmings running off a cliff.  I shake by head when I hear this because people still don't understand why Apple commands this kind of loyalty.  So let me state the obvious.  It's not the specs.  To borrow from James Carville, "It's the usability, stupid!"

There are many smart phones with better cameras, faster processors and ultra fast 4G networks rocketing them ahead in the specs department.  However, in the interest of usability, Apple does the following things:

1. Control the hardware.  Apple's software is stable because it's designed to run on a limited line of devices, whereas Google and Microsoft license their software to just about anyone.  If you can't guarantee the hardware, you can't guarantee the software.
   
   
2. Control the user experience.  Apple will not allow network providers to dictate device features or create "exclusive" phones for their service.  True, Apple and AT&T did originally have an exclusivity agreement, but it was truly exclusive so the user experience never suffered. Samsung learned this the hard way and is now pulling in the reigns on Verizon, AT&T and Sprint.  Network providers may retaliate by lowering smart phones subsidies, but at least we'll get a common experience out of it.
   
   
3. Offer easy OS upgrades for free and not too frequently.  This will ensure a quick adoption rate. 
•  I can't speak for Google, but Microsoft got it all wrong and is now taking a page from Apple's playbook.  It took Microsoft over 10 years to get 51% of their users off of Windows XP because they were charging hundreds for features that made us yawn.  Windows 8 pricing?  $40.  
• Apple offers updates every one to two years.  Every time I turn on my Xbox 360 or reboot my Win7 machine, I'm getting forced updates with no idea what benefit I'm getting.
• Windows 8 distribution is now more like the iTunes Store.  Nix the DVD, and offer downloads
  
  
4. Design is dictated by usability and vice versa.  Engineers are critical for some things, but leave them out of the design discussions unless you want key features hidden 8 menu layers deep.
   
   
5. Apple only introduces features that are mature and stable.  A fast network and 200 megapixel camera are useless if your apps and OS crash. Apple also control's apps through a tight QA process. 
   
   
6. Don't compete on price.  It cheapens your image and invites the crowds that want to jailbreak your product to death.  These folks then kick and scream to their friends about how it doesn't work.
   
   
7. Protect your intellectual property.  Everyone takes the pinch zoom for granted and many claim it should be not be patentable because it's "intuitive", but that's exactly why it deserves a patent!  Someone came up with that idea and they deserve to be paid.  Apple also goes aggressively after unlicensed third party accessory makers.  This may increase the price, but it ensures that these things have some quality standards.
   
   
8. Eliminate competition at the point of sale.  Apple's own stores sell only Apple products.  Now they control the manner in which the products are displayed and have expert help who are qualified to answer questions.  Despite a lack of choice for consumers, the stores and the gadgets are exciting enough to draw people in.

How to Deliver Training

Technical knowledge and training are two separate skills.  Case in point, last week I had training on SQL Server Reporting Services (SSRS).  There were two presenters and both were very technically knowledgeable, but their training abilities left something to be desired.  If you are a company putting on training or you are a trainer yourself, take heed to these pointers:

• Know your audience. The training isn't about you - it's about them.  Is your audience developers, project managers, architects, engineers, operational support, business lines or executives?
• Have an agenda.  Do not wing it!  Include subject matter and breaks.  One tried and true formula is intro, body, and conclusion, also known as tell them what you're going to tell them, tell them and then tell them what you told them.
• Always tag team.  
• One person can train while the other controls the slide deck and adjusts lighting and sound. 
• This is the best way to keep your audience engaged.  It is harder for people to zone out if there are alternating voices, personalities and bodies.
• Instructors should be apart from or roam within the audience.  This ensures that  they can receive nonverbal cues from the audience.
• If you are the only one training, move around a lot, animate your vocal patterns and involve the audience as much as possible.
• Make sure you can be heard.  Get a microphone if you're in a room with a capacity over 10.
• Announce how questions will be handled.  
• Roaming instructor
• Ask in real time
• Q&A section
• Powerpoint is optional.  A few large visual aids printed at Kinko's can be just as effective.  If you do use Powerpoint, use it for high level points.  It's your job to fill in the details.  No more than 50 words on a slide.
• Have a common theme. Use similar color patterns, branding and terms for that extra polish.
• Check out the venue in advance.  Get a sense of the room's size, amenities and potential obstacles.  This will give you a sense of comfort with your environment and prevent you from not being able to find your place on game day.
• Know your configurations.  Sometimes the room is fixed (stadium seating) but many times it can set up to need.  Lectures are great for disseminating expert knowledge or policy decisions, horseshoes work for team meetings, analytics, and video conferencing and roundtables work for collaboration and negotiating. New virtual classroom software is also quite sophisticated and commonly used.
  
  
• Make sure you have what you need.  Will training PCs be locked?  Will you need Internet access?  Test setting up the technology and doing a dry run.  Have the contact info for the AV and Service Management teams.
• Get personal.  Make your contact information available.  Have your audience wear name tags or use tent placards and then call people by their names. 
• Break.  At least 10 minutes every 90 minutes of training and one hour long break if over 6 hours.  Provide drinks and snacks if over four hours.
• Dress for success.  Grooming and hygiene should also go without saying.
• Be flexible.  No matter how much you prep, something will go wrong.  Diffuse it with humor and move on to Plan B until the issues can be resolved.
• Ask for feedback.  Get some data of your own to analyze and improve your training skills!

My Journey BegIns

After two years of studying, networking, and interviewing on the subjects of Enterprise Resource Planning, Customer Relationship Management, and Business Intelligence, I have accepted a position with my current employer supporting BI tools.  I am very exciting and I can't wait to help mature this very important field: a field that is new and interesting but one that will also bring real results to the business.  There is no better way to get hands on experience than to support Production systems and I look forward to working with the engineering and consulting folks of BI as well.

Why Document Libraries are Better than File Systems

He's not Immortal. He just Designs Universes.

I recently interviewed Business Objects universe designer and reporting expert David Hansen. Here's what he had to say:

Which school did you get your degree from and what did you major in?

I attended Carroll University from 2005 to 2009 and graduated with a bachelor’s degree in software engineering and a bachelor’s degree in mathematics. During my time at Carroll I played on the D3 soccer team there. My senior year we made it to the second round of the NCAA D3 soccer championship. For me, college was a great experience because I was able to experience both the academic and athletic sides of college life.

What area of Business Intelligence do you consider yourself a subject matter expert in?

When it comes to Business Objects, I’ve got a lot of experience in just about everything. Business Objects requires a universe to be built before you can build any reports. This universe is built “on” a database, so you have to know your database before building the universe. The universe follows the same rules as a database does, so you’ll need some experience in setting up a database with relationships and foreign keys before building a universe. Once the universe is built, you can use the Business Objects report designer tool to start crafting some reports. What is nice about the report designer tool is that you can override the sql query the report automatically creates to write your own, more complex query if you are good with sql.

Do you enjoy what you do and why do you feel it is important?

I love what I do, and I’m so lucky to have found such a fun, interesting career path. Creating reports and dashboards using queries and code is right up my alley. With degrees in mathematics and software engineering, I’m able to accomplish both things I love on a daily basis. And it isn’t worthless, either. For instance, the movie Moneyball recently came out, and that movie was about metrics in baseball. Using mathematics and numbers to determine strengths and weaknesses in players and build a strong team is close to what I do. Instead of making business decisions on gut feelings, I try to build reports for executives so they can make decisions based on numbers and facts. I’m helping them run their business as efficient as possible. If I can help people understand what is really going on with their area using reports and dashboards, the company can work more efficiently.

I’m hearing a lot about dashboards and dashboarding. What is this?

A dashboard is a user interface that is designed to give a quick, but deep, overview of what is going on within a certain area of business. A dashboard should be easy to read and understand, and it should be able to be able to convey all relevant information within the first two minutes of seeing it. This is different than a scorecard or report. A scorecard lists a bunch of numerical values for different metrics in a business area, and a report lists all the information a business area might have. A dashboard differs from these because it creates charts and visuals in such a way that it can tell you how your business area is doing without jumping into all the numbers of a scorecard and a report.

What challenges do you face when trying to procure data for analysis? How do you overcome this?

One of the biggest challenges when it comes to producing data is making sure that the data is accurate. In a perfect world, any data collected is relevant and carries the same weight as everything else. Unfortunately, you run into situations where business areas try to “cheat the system” to improve their numbers without improving their quality of support. To get around this, you can try to write rules to get rid of outliers and data points that will have an adverse affect your reports and dashboards.

If you do development work which language/environment do you prefer to develop in and why?

During school, I got to taste a number of different languages: Visual Basic, C++, C#, Java. However, I think the most important language in my work every day is VBA (Visual Basic for Applications). VBA is pretty much a watered down version of Visual Basic. However, it is still powerful and dynamic enough to do some great things. What makes VBA so influential is that it lies behind all the Microsoft products (press Alt + F11 to open it). Using VBA, you can do a wide array of things right from Excel, like pulling data from a database and producing a dashboard, or simply copying and saving files from one folder to another. One of the things that makes my job so much fun is a lot of people hack and slash a report together in Excel, and it takes them hours upon hours to create. But then I come in, write some VBA code behind their report, and allow them to click a button that creates their report in seconds. The look of astonishment on their face is awesome.

Do you feel Business Intelligence Analysts will become increasingly necessary in the years to come or that BI will become more of a commodity as DBMS, ETL and reporting tools mature?

The one thing that Business Objects has going for it is that you can create ad-hoc reports with ease. Obviously you’ll need a knowledgeable person to set up the Business Objects universe. However, once that is done, the learning curve to building reports isn’t that steep. That’s where I think Business Objects shines; you don’t need an expert in report building to create a report for you. Now, there are more complex reports that require a little more knowledge than your average bear. Yet, most of the simple, ad-hoc reporting can be completed by the end user, which frees up the time of the data experts and allows them to focus their time on higher level reporting. The end user is able to gather his or her data whenever he or she wants easily and effortlessly, and that is important for a tool.

For businesses that don’t have a business intelligence strategy, where do you think they should start?

The starting point is building a well designed database and gathering data. Bring someone in who knows a lot about data integrity and database design. Once the database is running, start collecting data! Once you have your preliminary data, you can start to evolve your thinking about a company. Maybe at first you just want to see call times for your support staff. After collecting data for a couple months, maybe you want to see what types of calls have which call lengths. Then, maybe you want to see what time the calls are mostly coming in so you can determine what staffing you need. Once you have a starting point, you can start to grow and learn more about your company and how business actually works!

What hobbies do you have?

I’ve got a ton of hobbies and am a very active person. I do soccer, curling, tennis, golf, bowling, chess, video games, fishing, hiking, traveling, board games, card games… I’m a competitive person who enjoys any type of game, so I do just about anything! Except basketball. I’m terrible at basketball.

Curling is quite a different hobby. Do you need special shoes for this sport?

I love curling. It’s addicting and fun. Some people describe it as chess on ice. And it’s a drinking sport. Who doesn’t love drinking some beer while throwing rocks down a sheet of ice? And actually, you don’t need special shoes for curling. When you shoot a stone, you need the least amount of friction as possible. Ergo, when you shoot, you put a special sole over your tennis shoe that creates a lot less friction. After you shoot, you take it off so you can re-gain most of the stability on the ice. I highly suggest that a person try curling, and you can always look online for more information regarding the sport!

Why Mergers Happen in IT

Prices can be set according to the benefits a seller thinks they provide, or they can be set to optimize profitability. At least option two is market driven, but that often leads to nothing but a pricing war. So if vendors want to avoid commoditization, they can either differentiate their product in a way that gets through to customers or they can reduce their customer's choices. Everyone knows that monopolies can charge whatever they want. That is why we dislike them and why even Adam Smith's invisible hand needs to be smacked on the knuckles like an ornery nun's ruler from time to time - a smack we call antitrust.

Sometimes mergers are good. They can allow a company with a broader reach to deliver innovative services previously only delivered by a more nimble upstart. They can also coalesce standards and mature products in a world previously chaotic from over-choice. This is usually early in a product life-cycle, however. Once a market matures and products are at risk of becoming commodities, mergers become the defensive maneuver to protect market share and profitability rather than a way to deliver better products and services.

This is where it can get tricky for IT managers. The easiest time to simplify to a preferred vendor is when the market is mature. This is also the ripest time for a "mad merger" to happen; the kind that happens simply to reduce choice. This mad merger can be especially bad if your vendor is acquired by another company with no previous relationship with you. Your best option may be to continue your efforts to simplify along the routes of preferred vendors, but keeping a 4th and 10 playbook in your back pocket just in case this mad merger ever happens. Consider things like 3rd party support for for your environment which will give you time to prepare your next move. You may also consider a backup vendor you can go with. This may sound unimaginable, but it is far easier to migrate a simple well oiled environment if you want to than to manage a kludgy patchwork of interconnected chaos because you have to. So in a nutshell, be mindful of the mergers and have a plan just in case, but do not stop simplifying!

Why BI is Primed to be BIg

The ideas behind Business Intelligence (BI) go back decades. Online Transaction Processing and Decision Support Systems are the predecessors of something whose time has come. BI is primed to be the single competitive advantage that makes or breaks companies. Here are five reasons this is true:

1. Simplifying the Environment - IT environments are way too complex. There are too many applications running on too many systems being supported by too many people. This house of cards is being held together by a patchwork of home grown solutions and over customized out of the box middleware. Sound familiar? Reducing complexity can seem daunting and this keeps many shops paralyzed. The good news is that executives are getting the memo. The tough economy has taught us that less is more. Plus, the more you simplify, the easier it becomes to simplify more. Be tough on taking things out of the environment. Don't let politics dictate which pet technologies get to stay.
2. Virtualization - Virtual machines have drastically shortened infrastructure planning. They also allow data warehouse environments to be set up quickly for project managers to see how technologies will play together for their pilot or proof of concept before full scale Production deployments. Computing resources are also allocated more efficiently, allowing unused memory and CPU to be assigned dynamically to whatever needs it. Cloud computing will only accelerate this trend.
3. Database abstraction - New tools and programming practices are insulating users from the database details, freeing IT to set up data in business terms one time rather than writing reports for each type of decision that needs to be made. This empowers business users to get the answers they seek, creating BI analysts in their respective business areas of expertise.
4. Social networking - There are new sources of data available directly from existing and potential customers. Business Intelligence systems can meld customer interests, friend networks, and buying behaviors from traditional point of sale sources to generate new leads, improve forecasting, and guide product development. These days, product life cycles are short, public sentiment can turn on a dime and competitors can come out of nowhere. To stay on top of things, intuition and fact based decision making need to be partners.
5. Mobile Computing - Today's business leaders and sales people want their information from anywhere on all of their devices. This creates a headache for information security, but real opportunities for BI since better data is created when it's entered in real time and tagged with location metadata.

These five things will enable BI to be big, but that won't necessarily make it big for you. For you it should start with goals (start with your mission statement and go from there). Once you know the goals of your organization, you're equipped to look for answers that can enable you to achieve those goals. Along your data mining journey, you will come across many interesting things, but if you don't use your goals as the foundation, these interesting nuggets can just as easily lead you astray, so stay focused and happy mining!