The proposal process is always an interesting time for us. It’s a little like dating. Both parties are taking some time to get to know one another before deciding if you want to get married or part ways.

Our clients range in size from 5 to 1000 employees, so every engagement is different. Not only in regard to the nature of their network, IT infrastructure, and business, but also what type of personnel you will be working with. It can range from companies with no internal IT staff at all to companies with robust IT departments.

In this particular situation, it was a setup we see too often – one single internal IT person. For the sake of the story, we will call him “Chuck”. We are always slightly more aware when we are looking at a scenario with a single IT resource since there are a number of risks associated with it and one of them reared its ugly head this time.

We had been speaking with the company for a couple of weeks and were preparing a proposal with the understanding that Chuck would continue on his current role as IT manager. Our primary role was to handle things that were outside of Chuck’s core competency or related to the new out-of-state locations the company was opening. They were growing pretty quickly and their IT was rapidly becoming more complex.

We were really close to pushing a final proposal when we got a panicked call from the prospect – Chuck had quit. It was a Tuesday and Chuck let them know his last day would be Friday. We immediately kicked into scramble mode. Our proposal was shot, because it was all based on Chuck being there. Our immediate goal was to download as much of the history that Chuck had about the environment before he left. He had been managing their systems on his own for several years and most of the configuration information about the infrastructure lived solely in his head.

This scenario was tough enough but then Wednesday rolled around and there was no sign of Chuck. No answer when we or the client called. Same went for Thursday and Friday. Chuck had disappeared and taken his knowledge and the history of the environment with him. Admin passwords, configurations, reasons for architecture decisions, and knowledge of proprietary systems were all gone! Add to it that there was minimal documentation of any kind, and we and the client were left flying blind.

As a result, we had to start from scratch requiring many extra hours reverse engineering how things were configured; much of which could have been avoided with a professional hand-off from Chuck or detailed documentation. The result was unnecessary business disruption and pretty big expense to the client. A key lesson here is you need to think hard about who owns the mindshare of your technology. Make sure if they disappear – your IT doesn’t go with them.

We have all experienced that situation where your spouse or a strange noise has roused you out of bed in the middle of the night. It normally turns out to be nothing. However, on day in March, it was much more than that for a Buffalo, NY man.

After putting on his robe and going to the top of the stairs, he was greeted by 7 armed men with assault weapons that had just broken down his door. They were screaming at him to get down as he screamed back demanding to know who they were. Grabbed and thrown down the steps he found himself face down in his own living room where the officers called him things like “Pedophile” and “Pornographer.”

Insisting that he didn’t know what they were talking about didn’t prevent them from going through his home and confiscating every laptop, desktop, iPad and mobile phone on the premises. As he lay there on the ground hurt and confused, it became clear to him what had happened.

That wireless router. He had purchased it recently and put it on his home network but, not being very tech savvy, he had gotten frustrated and when he couldn’t figure out how to set up the encryption and password protection and just decided to leave it open.

Through that open wireless connection, someone within reach of the signal had downloaded thousands of images of child pornography. It took Customs Officers three days to determine that he was telling the truth, and they eventually returned his devices and issued him an apology.

Customs Agents had traced the violation back to his router’s unique IP address and initiated the early morning raid, literally and figuratively turning his world upside down.

They later caught the man that had hijacked the wireless signal. The crime they couldn’t charge him with, however, was accessing the wireless signal. The courts have yet to make a determination, but as it stands, leaving your wireless signal open and unencrypted is implied permission for anyone that gets within range of the signal to use it.

It is always important to protect your internet access for many reason, this being one of the scarier ones. There have been many circumstances where we see companies employing these types of consumer grade devices in the company environment, rather than investing in a proper business class router and wireless access points that are properly configured and encrypted for maximum protection. Certainly not best practices.

Protect yourself and your internet connection. I can think of a lot of acceptable ways to be woken up, but armed Federal Agents isn’t one of them.

In almost all cases, when we are in the proposal process with a potential client, we like to do a physical walkthrough of their current data closet or server room. It’s a very important part of the discovery for us. As you might imagine, we always see a vast array of situations and circumstances (many of which will likely find their way to this column).

As we began to explore this particular environment, we were very pleasantly surprised at the situation we found. They had a nice server rack and business-grade hardware. Also, there was a supplemental cooling unit and raised floors. These are things we don’t see a lot when doing discovery but would definitely consider best practices for an on-premise environment.

The only slight issue we found on the eye test was some pretty haphazard cabling in the back of the rack. As we were poking around and looking at the cable structure, one of our engineers noticed a single cable that seemed to be going, well, into the floor. Certainly an odd sight considering most of the cables went to the switch above it. Naturally, we questioned the CEO and IT Manager as to the end point for the rogue cable. The response: “I don’t know.”

We decided to employ one of our more sophisticated analysis procedures: pulling slightly on the cable to see if it was loose. As we pulled, we heard a small scrape and felt something sliding under the flooring.

A quick lift of the floor tile revealed a small PC that was living underground and wired to the network. Turns out it was an employee’s hardware with their personal web server, piggybacking off of the corporate network so they could access a static IP address. A nice benefit for the employee, but not so good for the company, considering the wide open anonymous FTP running on that personal server is a major security risk. Never mind the whole bit about unauthorized use of company resources.

Obviously, company networks carry a lot of passengers. Just make sure you know everyone that is on the bus.

About 4 months ago, we pitched a client on out IT services. It was a pretty typical meeting. They had a modest infrastructure: a couple of servers and some network equipment running and no real ongoing maintenance to speak of. After the meeting, we had outlined a number of things that would improve the performance and reliability of their environment and, perhaps most importantly, make it more secure.

The sticking point in closing the deal was the ever-present, “Well, we have an IT guy.” (Who we will refer to as ‘Chuck’) Certainly we are fans of any company having internal staff to support their technology. Unfortunately, often people don’t realize is that there are many, many different disciplines within IT and there really isn’t a magical “one guy” that knows it all. What that means is that they are doing the things that they are good at, but, unfortunately, they are also trying to do the things that they aren’t very good at. In this case, they didn’t feel like they wanted to use a firm because Chuck had everything under control.

As we do in those situations, we gave them a card and asked to please let us know if we could be any help in the future. As you probably guessed already, the phone rang about 2 months later. Their network had slowed down to a snail’s pace and was affecting the ability of people to get work done and Chuck couldn’t seem to isolate what was causing the performance issue. They asked us if we would take a look.

We had an engineer remote into their system and poke around. Thirty minutes of analysis later, he made a pretty interesting discovery and let us know what he had found. After a quick discussion on the best approach to tell the client, we made the call.

“There’s good news and bad news,” we said. “The good news is that we can have the problem corrected within the next 2 hours. The bad news is that one of your corporate servers has been compromised and is currently hosting an adult site.” I wish I had the right adjectives to describe the level of silence from the other end of the line, but it isn’t coming to me at the moment.

Naturally, the first question was “How did this happen?” In this particular case, Chuck had not properly configured their firewall and had failed to maintain security patches on their email server. The hacker in question was fairly easily able to bypass the misconfigured firewall and simply go about dropping all the files onto their system, and, voila, the customer was, without their knowledge, in the adult film business.
Sounds crazy, but a company’s technology has numerous moving parts and requires a lot of expertise to manage, maintain, and keep secure. I hope none of you find yourselves unwittingly in the adult entertainment business…